Communication system, information processing apparatus, terminal device, information processing method, and program

Abstract

To provide a technique which enables a participant to easily recognize information useful to him or her by displaying information based on a designated range of language information.SOLUTION: The present invention provides a communication system in which an information processing apparatus and one or more terminal devices communicate with each other over a network. Each of the terminal devices has a first display control unit which displays language information based on information inputted to the terminal device, in a conversation accumulation area 101 (first area) of an information display screen 100 of the terminal device in time series. The information processing apparatus has an analytic information acquisition unit which acquires analytic information obtained by analyzing a designated range of language information out of language information based on the information inputted to the terminal device. The terminal device has a second display control unit which displays the analytic information acquired by the information processing apparatus in a whiteboard work area 102 (second area) of the information display screen 100.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a communication system, an information processing apparatus, a terminal device, an information processing method, and a program.

Background Art

[0002] In a communication such as a meeting, the moderator may write down the remarks of the participants on a whiteboard or the like, and share the content of the discussion by visualizing the language information. Participants in the meeting may extract and transcribe important keywords, or enclose them in a frame. In addition, in order to assist the work on the whiteboard, a technique is known in which the remarks of the participants in the meeting are converted into language information by voice recognition and displayed.

[0003] As an example of a technique that utilizes such voice-recognized language information, the invention described in Patent Document 1 has been proposed. Patent Document 1 discloses a display control system that performs keyword extraction from language information obtained by textifying the remarks of participants, executes a search process for searching for data based on the extracted keywords, and displays the search results including the language information and images searched by the search process in different display areas.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, since the target range of the language information for executing the search process cannot be specified, search results based on the remarks of the participants are continuously displayed, and there is a risk that the display area of the search results becomes cluttered. Therefore, there is a problem that it is difficult for the participants to grasp useful information.

[0005] In view of the above problems, the present invention provides a technique that enables participants to easily grasp useful information for the participants by displaying information based on language information within a specified range.

Means for Solving the Problems

[0006] In view of the above problems, the present invention is a communication system in which an information processing device and one or more terminal devices communicate via a network. The terminal device has a first display control unit that displays language information based on information input to the terminal device in a time series in a first area of an information display screen of the terminal device. The information processing device has an analysis information acquisition unit that acquires analysis information obtained by analyzing language information within a specified range among the language information based on the information input to the terminal device. The terminal device has a second display control unit that displays the analysis information acquired by the information processing device in a second area of the information display screen.

Effects of the Invention

[0007] Since information based on language information within a specified range can be displayed, participants can easily grasp useful information for them.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Embodiments for Carrying Out the Invention

[0009] Hereinafter, as an example of an embodiment for carrying out the present invention, a communication system and an information processing method performed by the communication system will be described with reference to the drawings.

[0010] <Overview of Operations> In a communication scenario, it is known that efficient and accurate information transmission becomes possible when visual information such as illustrations and icons is added to language information. For example, in a meeting held to present new plans or ideas, it is effective to utilize visual information for communication among multiple participants. To streamline communication using this visual information, a whiteboard application that enables collaborative editing by multiple participants online is known.

[0011] For example, a first online whiteboard service provided on the Web is known. The first online whiteboard service displays in real time, in a history area, text data (voice recognition results or keyboard input results) input by participants in a meeting called a talk log by a Web application. Further, the Web application displays an online whiteboard screen called a workspace. Participants in the meeting can arrange the text data in the talk log on the online whiteboard screen in the workspace by pressing, dragging, etc., enabling multiple people to collaborate online while visualizing the communication process.

[0012] However, in the first online whiteboard service, in order to place the text data of the talk log of the meeting participants on the online whiteboard screen of the workspace, manual operations by the participants such as pressing and dragging the text data of the talk log were required. For example, in a meeting where an active discussion (the talk log scrolls quickly) is held, it is difficult for the meeting participants to extract useful information from the talk log, and it was necessary to get used to using it. Also, even for experienced participants, there was the inconvenience that their attention was diverted to extracting statements from the talk log, resulting in sparse participation in the discussion of the meeting.

[0013] Also, a second online whiteboard service having an assist function using AI is also known. The assist function provides functions such as "similar ideas proposed by the generative AI are added to the whiteboard", "a summary is created by selecting and executing multiple pieces of information", and "an image associated with the selected text is generated" when the meeting participants select and execute the text input to the whiteboard.

[0014] However, in the second online whiteboard service, manual operations by the participants such as the meeting participants selecting text data were required.

[0015] Thus, in the prior art, the intervention of the participants was necessary to extract useful information from the text data input by the participants. For example, it was not possible to automatically summarize ideas from the language information input in real time by the participants attending the meeting.

[0016] Therefore, the communication system of this embodiment automatically analyzes a specified range of language information based on the information input during communication, and automatically places the analysis result in the shared whiteboard work area. Since the range of language information is specified, the displayed information can be restricted. Participants can grasp only useful information. In addition, the communication system extracts ideas included in the communication and summarizes the flow of the discussion. By doing so, the burden on the participants in the communication can be reduced.

[0017] [First Embodiment] [Example of Workspace Display During Communication] Referring to FIG. 1, an outline of the operation of the communication system of this embodiment will be described. FIG. 1 shows an information display screen 100 displayed by the terminal device during a meeting. As shown in FIG. 1, the information display screen 100 has a conversation storage area 101 (an example of a first area) and a whiteboard work area 102 (an example of a second area). The conversation storage area 101 is an area where the history of language information based on voice data obtained by voice input of the content of conversations by the participants in the meeting and converted by voice recognition, or language information based on text data input by keyboard, is displayed. The whiteboard work area 102 is an area where participants can display information based on the language information in the conversation storage area 101, handwritten data, directly input language information, or various images, in the same manner as a normal electronic blackboard. Note that it is preferable for the terminal device to have a touch panel for input of handwritten data, but a normal display may also be used. In this case, a participant who operates the terminal device (hereinafter sometimes simply referred to as a participant) uses a mouse to perform handwritten input and the like.

[0018] For the sake of explanation, the language information 201 to 204 in the conversation storage area 101 is shown below. The language information 201 is "Shall we start with self-introductions? Well, if you'll allow me to assign a bit of an order, could we start with Mr. Naito first?" The language information 202 is "Yes, well, I'm from the camera design department. This time, I'm here as the representative. I really hope we can have a fun project with all of you. Nice to meet you." The language information 203 is "Then, I'll leave it to Mr. Mori next. Yes, um, I'm here representing Tri-Bus, which operates new business programs within the company. Nice to meet you." The language information 204 is "Nice to meet you. Nice to meet you. Yes, then, I'll ask you this morning. Yes, I'm Ikusawa. Well, I'm in the hardware department for office equipment. I'm in charge of the strategy for the multifunction printer business. Nice to meet you."

[0019] The information processing apparatus described below analyzes these language information automatically without the intervention of the conference participants. For example, it is conceivable to use a large language model for the analysis. In FIG. 1, in the whiteboard work area 102, the topic 211 of the language information 201 to 204 and the topics 212 to 214 of the language information 201 to 204 are respectively displayed in message boxes. A message box is a display component where characters are arranged. In this way, the information processing apparatus can automatically display the topic 211 and the topics 212 to 214 from the plurality of language information 201 to 204 and the language information 201 to 204 of different speakers.

[0020] The topic 211 is the result of the large language model analyzing the language information 201 to 204 to analyze what topics are being discussed. The topics 212 to 214 are the analysis results extracted by the large language model analyzing the language information 201 to 204 to analyze what topics are being discussed.

[0021] Also, for the sake of explanation in FIG. 1, arrows are shown from the language information 201 to 204 to the corresponding topics 212 to 214. These arrows are not displayed during actual use (they may be displayed). As shown in FIG. 1, it can be seen that the topics 212 to 214 of the language information 201 to 204 are grouped in the whiteboard work area 102. Further, the information processing device can arrange three topics 212 to 214 in parallel below the topic 211 and display them on the terminal device by fitting the topic 211 and the topics 212 to 214 into a preset hierarchical structure. Therefore, it has become easier for the participants to grasp the relationship between the topic 211 and the topics 212 to 214.

[0022] <Regarding Terms> Language information refers to information composed of languages such as Japanese and English. In this embodiment, language information includes not only the words spoken by the speaker that have been speech recognized, but also cases where input is made from a keyboard or the like. Language information includes information that can be displayed by a computer based on character codes such as characters, figures, numbers, alphabets, etc.

[0023] Accumulating in time series means that the language information is displayed in the order of speech, and it is possible to arbitrarily view past language information.

[0024] Analysis of language information means generating meaningful information based on the language information. The generated information is the analysis result, which is referred to as analysis information in this embodiment. The analysis information may be included in the language information or may be obtained from the outside based on the language information. The analysis information may have a conceptually narrower language information or may include concepts not included in the language information. The analysis information may include not only language information but also figures, images, URLs of web pages, etc.

[0025] The working area is an area on a computer screen where participants can edit the arrangement and content of language information and content. As an example of a working area, there is an area that can be used like an electronic whiteboard. An electronic whiteboard is a device that can use a touch panel to write handwritten data in the same way as a conventional whiteboard and can project the screen and data of a PC or the like. Note that the whiteboard may be referred to as an electronic blackboard, an electronic information board, an electronic whiteboard, or the like.

[0026] Content is a display object displayed in the working area. Content can be, for example, language information, illustrations, graphics, tables, the screens and data of a PC or the like, information on the Internet, etc., as long as it can be displayed on a computer and is visible to participants. Also, the analysis information displayed in the working area is also content.

[0027] The analysis of language information means generating meaningful information based on the language information. The generated information is the analysis result, and in this embodiment, it is referred to as the AI analysis result. The AI analysis result may be included in the language information or may be obtained from the outside based on the language information. The AI analysis result may conceptually narrow the language information or may include concepts not included in the language information. The AI analysis result may include not only language information but also graphics, images, URLs of web pages, etc. In this embodiment, by analyzing the language information, for example, one or more of major items, key sentences, illustrations, and categories are generated. Therefore, the AI analysis result is one or more of major items, key sentences, illustrations, and categories.

[0028] <System Configuration Example> Figure 2 is a system configuration diagram showing the system configuration of the communication system 5. The communication system 5 includes a plurality of terminal devices 1a to 1c and an information processing device 3 that can communicate via the network 2. Further, in Figure 2, the generation AI system 4 is connected to the network 2. Note that any of the terminal devices 1a to 1c is referred to as the "terminal device 1". The terminal device 1 does not necessarily need to be always connected to the network 2. The number of terminal devices 1 is not limited to three, and two or more are sufficient for communication. However, even if there is only one participant in a meeting, the communication system 5 can be used.

[0029] The network 2 may be constructed using one or more of the Internet, a mobile communication network, a LAN (Local Area Network), etc. The network 2 may include a network by wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0030] The terminal device 1 is an information processing device operated by a participant. The terminal device 1 transmits voice data uttered by the participant to the information processing device 3. The terminal device 1 receives the language information obtained by converting the voice data into characters by the server and displays it in the conversation storage area 101. The terminal device 1 repeats this process. A plurality of terminal devices 1 operated by participants participating in the meeting are connected to the network 2, and by each terminal device 1 performing the same process, the respective language information input by each participant is displayed in the conversation storage area 101 in chronological order.

[0031] The terminal device 1 may be, for example, a desktop PC (personal computer), a notebook PC, a smartphone, a tablet terminal, etc. that operate on common operating systems such as Windows (registered trademark), MAC OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc. In addition, the terminal device 1 may be any device on which applications such as a web browser can operate.

[0032] Furthermore, the terminal device 1 may also be an output device such as an electronic blackboard or a digital signage, a HUD (Head Up Display) device, an industrial machine, a medical device, a network home appliance, an automobile (Connected Car), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), etc.

[0033] The information processing device 3 is an information processing device that fulfills the function of providing information and processing results in response to requests from the terminal device 1. Such an information processing device is called a server. In the communication system 5 of this embodiment, the information processing device 3 may provide images such as illustrations corresponding to language information to the terminal device 1 in addition to performing the above speech recognition. Note that speech recognition is also possible on the terminal device 1. Also, the information processing device 3 transmits language information, etc. to the generation AI system 4 and receives analysis information (for example, the above-mentioned topics and subjects, etc.).

[0034] The information processing device 3 preferably has a function of a web server. A web server transmits various information such as HTML documents existing in the web server to the terminal device 1 in accordance with HTTP connected from the terminal device 1 based on the URL instructed by the web browser which is a client. Further, the information processing device 3 can transmit a web app to the terminal device 1. When the web browser executes the web app, the web browser and the web server communicate with each other. A web app is a program written in a programming language (for example, JavaScript (registered trademark)) that operates on the web browser. By the cooperation of the web app and the program on the web server side, the web browser can provide various services to the participants. On the other hand, an application that is not executed unless it is installed in the terminal device 1 is called a native app.

[0035] The information processing device 3 may support cloud computing. Cloud computing refers to a usage form in which resources on a network are used without awareness of specific hardware resources. Cloud computing has forms of SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service), and any form may be used. Further, servers corresponding to the functions of the web server, voice recognition function, illustration search function, etc. possessed by the information processing device 3 may exist separately.

[0036] The generative AI system 4 provides a service that enables participants in a meeting to have a conversation with artificial intelligence (AI) in a conversational format. As such a generative AI system 4, a system using large language models (LLMs) is known. A large language model is a natural language processing model trained using a large amount of text data. The generative AI system 4 incorporates a vast amount of text in advance and learns knowledge from it through deep learning, reinforcement learning, etc. By utilizing this knowledge, the generative AI system 4 can perform various natural language processing tasks. For example, natural language processing tasks include conversation, text summarization, novel creation, translation, creation of meeting minutes and reports, text checking and proofreading, and program generation.

[0037] The generative AI system 4 can be directly used by the participants in the meeting or can be used via the information processing device 3. For example, a participant operates the terminal device 1 to input a chat message and causes the terminal device 1 to send the chat message to the generative AI system 4. The generative AI system 4 generates an appropriate response message for the chat message and sends it to the terminal device 1. Alternatively, a participant operates the terminal device 1 to input a chat message and causes the terminal device 1 to send the chat message to the information processing device 3. The information processing device 3 calls the API (Application Programming Interface) published by the generative AI system 4 and sends the chat message to the generative AI system 4. The generative AI system 4 generates an appropriate response message for the chat message and sends it to the information processing device 3. The information processing device 3 sends the response message to the terminal device 1. In this embodiment, the generative AI system 4 may be used in either manner.

[0038] Also, in FIG. 2, the generative AI system 4 is shown separately from the information processing device 3, but the information processing device 3 and the generative AI system 4 may be integrated. In this embodiment, as an example, a form in which the information processing device 3 communicates with the generative AI system 4 will be described.

[0039] Note that an interface through which a participant inputs a chat message to the generative AI system 4 is called a prompt. Chat messages such as instructions and questions are input to the prompt. The chat message may be a conversation sentence or word composed of natural language spoken by a human, and in this embodiment, it is referred to as language information. Also, in the generative AI system 4, an image can also be used as a chat message. The generative AI system 4 that generates a sentence for a chat message may be called generative AI. In this embodiment, for the language information input by the participant, the generative AI system 4 generates a topic, a subject, etc.

[0040] As the generative AI system 4, systems using LLMs such as GPT-3, GPT-4, Transformer, and BERT (all trademarks) are known. As an example, ChatGPT (trademark) using GPT-3 or GPT-4 can be used for the generative AI system 4 of this embodiment, but systems using other LLMs may also be used.

[0041] <Hardware configuration example> Subsequently, with reference to FIG. 3, the hardware configurations of the information processing device 3 and the terminal device 1 will be described. FIG. 3 shows a hardware configuration diagram of an example of the terminal device 1 and the information processing device 3. The terminal device 1 and the information processing device 3 each have a hardware configuration of a computer 500 as shown in FIG. 3.

[0042] As shown in FIG. 3, the computer 500 includes a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, a media I / F 516, a motion acquisition device 517, a microphone 518, a speaker 519, and a camera 520.

[0043] Among these, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs used to drive the CPU 501 such as the IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as the program 500a. The HDD controller 505 controls the reading or writing of various data to and from the HD 504 according to the control of the CPU 501. The display 506 as a display means displays various information such as a cursor, menu, window, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. The external devices in this case are, for example, a USB (Universal Serial Bus) memory, a printer, etc. The network I / F 509 is an interface for performing data communication using the network 2. The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501 shown in FIG. 3.

[0044] Also, the keyboard 511 is a kind of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 512 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls the reading or writing of various data to and from the optical storage medium 513 as an example of a removable recording medium. Note that the optical drive 514 may be a CD, DVD, Blu-RayR, etc. The media I / F 516 controls the reading or writing (storage) of data to and from the recording medium 515 such as a flash memory.

[0045] The motion acquisition device 517 is a circuit that detects the motion of a participant and converts it into an electrical signal, and is a kind of input means. The detection of the participant's motion may be any one of an optical type that detects the reflection of light, a magnetic sensor attached to the participant, a mechanical sensor, one that uses the detection result of a magnetic sensor, or a method of analyzing the image of the participant, or a combination thereof.

[0046] The microphone 518 is a built-in circuit that converts sound into an electrical signal. The speaker 519 is a built-in circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voices. The microphone 518 and the speaker 519 may be built-in or externally attached.

[0047] The speaker 519 is a built-in circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voices.

[0048] The camera 520 is a type of built-in imaging means that captures a subject to obtain image data.

[0049] In the computer 500 applied to the information processing apparatus 3, the HDD controller 505 corresponds to the HD 504 and stores the program 500a. In the computer 500 applied to the terminal device 1, the program 500a is downloaded from the information processing apparatus 3 via the network 2 and stored in the HD 504. When the program 500a is read from the HD 504 and executed by the CPU 501, the functional configuration as shown in FIG. 4 described later can be expanded all at once during compilation or sequentially according to the progress of the process in the RAM 503. Also, in the computer 500 applied to the information processing apparatus 3, when the program 500a is executed by the CPU 501, the functional configuration as shown in FIG. 4 can be expanded all at once during compilation or according to the progress of the process in the RAM 503.

[0050] Also, in the computer 500 applied to the information processing apparatus 3, the configuration related to the user interface may be omitted, and at least one of the display 506, the keyboard 511, the pointing device 512, and the motion acquisition device 517 may be omitted.

[0051] <Functional Configuration of Communication System> Subsequently, the functional configuration of this embodiment will be described with reference to FIGS. 4 to 11. FIG. 4 is an example of a block diagram showing the functions of the communication system 5 divided into blocks for each of the information processing apparatus 3 and the terminal device 1.

[0052] <<Functional Configuration of Information Processing Device>> As shown in FIG. 4, the information processing device 3 includes a second communication unit 31, a conversion unit 32, a conversation content management unit 33, a content management unit 34, a translation unit 35, an analysis information arrangement unit 36, an analysis information acquisition unit 37, and a third communication unit 38. Each of these units included in the information processing device 3 is a function or means realized by any of the components shown in FIG. 3 operating according to instructions from the CPU 501 in accordance with a program expanded from the HD 504 onto the RAM 503. Further, the information processing device 3 includes a storage unit 3000 constructed by the RAM 503 and the HD 504 shown in FIG. 3.

[0053] The second communication unit 31 communicates with one or more terminal devices 1 via the network 2. The second communication unit 31 receives voice data or handwritten data from the terminal device 1 and transmits language information converted from the voice data or handwritten data to the terminal device 1. Further, the second communication unit 31 may also receive language information input by the participant using a keyboard from the terminal device 1.

[0054] The conversion unit 32 performs character recognition processing on voice data spoken by the participant or one or more stroke data (handwritten data) written by the participant, and converts it into language information. The entity of the language information is a character code. The conversion unit 32 recognizes characters (not only Japanese but also multiple languages such as English), numbers, symbols (%, $, &, etc.), graphics (lines, circles, triangles, etc.), etc. in parallel with the participant's speech and the participant's pen operation. Although various algorithms have been devised for the recognition method, in this embodiment, it is assumed that known techniques can be used, and the details are omitted. Note that algorithms suitable for the recognition of voice data and handwritten data are used respectively. The conversion unit 32 performs character recognition processing on voice data, which is voice information based on the speech of the participant, or one or more stroke data (handwritten data) written by the participant, and converts it into language information. In this embodiment, the participant's speech is referred to as "voice", and the voice converted into data is referred to as "voice data" for distinction, but in this embodiment, for ease of explanation, they are not strictly distinguished.

[0055] Note that the terminal device 1 may perform the conversion by the conversion unit 32. Further, the conversion unit 32 may be provided in a server independent of the information processing device 3. The terminal device 1 or the information processing device 3 transmits voice data or handwritten data to this server to obtain language information.

[0056] The conversation content management unit 33 stores the language information converted from the voice data of each terminal device 1 in the conversation content storage unit 3001. The conversation history is stored in the conversation content storage unit 3001 in chronological order.

[0057] The content management unit 34 manages the language information and illustrations displayed in the whiteboard work area 102. The content is managed in the content storage unit 3002 described later. Since the whiteboard work area 102 is shared by each terminal device 1, the content storage unit 3002 includes the content arranged by each participant. When the whiteboard work area 102 is different for each terminal device 1, the content storage unit 3002 manages the content arranged by each participant for each participant.

[0058] When the language used in the terminal device 1a is different from the languages used in the other terminal devices 1b and 1c, the translation unit 35 translates the language information from the language used in the terminal device 1a into the languages used in the other terminal devices 1b and 1c. Conversely, the translation unit 35 translates the language information from the languages used in the other terminal devices 1b and 1c into the language used in the terminal device 1.

[0059] The third communication unit 38 transmits a request message to the generation AI system 4 by calling an API published by the generation AI system 4, and receives a response message as a response. The request message includes language information, speaker information, date and time, etc., and the response message includes analysis information (topics, topics, summaries, divergence information, convergence information, keywords, etc.) generated by the generation AI system 4.

[0060] When the language information within a preset range is stored in the information processing apparatus 3, the analysis information acquisition unit 37 transmits, via the third communication unit 38, the analysis of the language information based on the analysis content set by the participant to the generation AI system 4. The analysis information acquisition unit 37 receives, via the third communication unit 38, the analysis information generated as the analysis result of the language information analyzed by the generation AI system 4. The analysis information acquisition unit 37 acquires the type of analysis information corresponding to the analysis content set by the participant.

[0061] The analysis information arrangement unit 36 determines the arrangement when arranging the analysis information acquired from the generation AI system in the whiteboard work area 102 using the arrangement information table in the arrangement information storage unit 3005 described later. The analysis information arrangement unit 36 determines, for example, the arrangement according to the type of analysis information generated by the generation AI system 4.

[0062] Note that, except for the second communication unit 31, part or all of the functions of the information processing apparatus 3 may be provided in the terminal device 1.

[0063] (Conversation Content Table) Next, the conversation content storage unit 3001, content storage unit 3002, illustration storage unit 3003, meeting management storage unit 3004, and arrangement information storage unit 3005 stored in the storage unit 3000 will be described.

[0064] FIG. 5 is a conceptual diagram showing a conversation content table. In the storage unit 3000, a conversation content storage unit 3001 configured by a conversation content table as shown in FIG. 5 is constructed. In this conversation content table, each item of language information ID, language information, input time, speaker, and like count is stored in association. · The item of language information ID is the identification information of the language information displayed in the conversation storage area 101. The conversation content management unit 33 numbers the language information IDs so as not to duplicate based on a certain number of characters or silent time. · The item of language information is the language information displayed in the conversation storage area 101 and is converted from the audio data by the conversion unit 32. · The item of input time is the time when the language information was input, for example, the time when the conversion unit 32 completed the conversion from voice data to language information. · The item of speaker is the identification information of the participant who spoke the language information as voice (hereinafter sometimes referred to as participant ID). The identification information of the participant is transmitted to the information processing device 3 together with the voice data. Note that the identification information of the participant is specified when the participant logs in to the information processing device 3. When multiple participants participate in the conference room but no login or the like is performed, there is a method in which the terminal device 1 recognizes the speaker by using a camera, voiceprint, or the like. · The item of number of likes is the number of times the like button has been pressed for the language information. The like button means a favorable reaction, and the number of likes is the number of favorable reactions.

[0065] (Content Table) Figure 6 is a conceptual diagram showing a content table. In the storage unit 3000, a content storage unit 3002 configured by a content table as shown in Figure 6 is constructed. The content is visual information arranged in the whiteboard work area 102. In this content table, each item of content ID, type, page, coordinates, size, font size, placement time, and language information ID is stored in association.

[0066] · The item of content ID is the identification information for identifying the content. The content ID is numbered by the content management unit 34.

[0067] · The item of type is the type of content, including handwriting, language information, graphics (such as squares, triangles, etc.), illustrations, tables, analysis information, etc. Handwriting is stroke data (coordinate point sequence). Language information is one or more characters, symbols, etc. (character codes) arranged in the whiteboard working area 102. Graphics are geometric shapes converted from handwriting data or language information such as triangles and squares. Illustrations are image data such as Jpeg, Png, Tiff, etc. that are retrieved by language information or directly arranged. A table is one-dimensional or two-dimensional tabular content. Analysis information is the content generated by the generation AI system 4 and indicates the type of analysis information.

[0068] · One screen of the whiteboard working area 102 is referred to as a page. The item of the page is its page number.

[0069] · The item of coordinates indicates the position of the content based on a predetermined origin in the whiteboard working area 102. The position of the content is, for example, the upper left vertex of the circumscribed rectangle of the content. Coordinates are represented, for example, in pixel units of the display.

[0070] · The item of size is the width and height of the circumscribed rectangle of the content.

[0071] · The item of font size is the size of one character set for the content whose type is language information. For language information, in addition, color, font, etc. may be set.

[0072] · The item of placement time is the date and time when the content was first placed in the whiteboard working area 102. The placement may be transferred from the conversation storage area 101 or directly written by the participant in the whiteboard working area 102. "First" means that even if the language information is decomposed by morphological analysis or the size of the illustration is changed, the placement time is not changed. This is to maintain the relationship with the input time of the language information in the conversation storage area 101.

[0073] · For the item of language information ID, when the language information or illustration is derived from the language information in the conversation storage area 101 (in the case of an illustration copied or retrieved from the conversation storage area 101), it is the language information ID of the language information. That is, the language information ID in the content table associates each content with the language information in the conversation storage area 101. In other words, no language information ID is set for content not derived from the language information in the conversation storage area 101. Also, when the type is analysis information, the language information ID indicates from which language information it is generated.

[0074] (Illustration Table) Figure 7 is a conceptual diagram showing an illustration table. In the storage unit 3000, an illustration storage unit 3003 configured by an illustration table as shown in Figure 7 is constructed. The illustration table is a table for converting keywords included in language information into illustrations. In the illustration table, each item of keyword, ID, and file name is stored in association. · The item of keyword is the name or designation of the illustration and is the character string to be searched. An illustration with a file name associated with a keyword that at least partially matches a word included in the language information is displayed on the terminal device 1. The keyword may be any of a word, phonetic character, ideogram, sentence, number, combination thereof, etc. · The item of ID is the identification information of the illustration. · The item of file name is the file name in which the illustration is stored.

[0075] An illustration table like Figure 7 is registered in the information processing device 3 in advance by an administrator or the like. Alternatively, without preparing an illustration table, the search unit 15 may search for illustrations on the Internet. Alternatively, the generation AI system 4 may generate an illustration corresponding to the word included in the language information.

[0076] In addition, the illustration may include a 3D model or a three-dimensional illustration. The 3D model or three-dimensional illustration is a virtual solid object in a three-dimensional space. The participant can view the 3D model or three-dimensional illustration from any direction out of 360°, and furthermore, the information processing device 3 can convert it into an illustration on a two-dimensional plane.

[0077] (Conference Management Table) FIG. 8 is a conceptual diagram showing a conference management table. In the storage unit 3000, a conference management storage unit 3004 configured by a conference management table as shown in FIG. 8 is constructed. The conference management table is information regarding a conference conducted by a participant using the information display screen 100, but information regarding a conference not using the information display screen 100 may also be registered. The conference management table has items such as a participant, a title (conference name), a start date and time, an end date and time, a location, and a password in association with a conference ID. · The item of the participant is a participant in the conference. · The item of the title represents the content of the conference, such as the conference name or the topic of the conference. · The item of the start date and time is the date and time when the conference is scheduled to start. · The item of the end date and time is the date and time when the conference is scheduled to end. · The item of the location is the venue of the conference, such as a conference room, a branch office name, a building, etc. · The item of the password is authentication information for participating in the conference and is distributed to the participants in advance.

[0078] (Layout Information Table) FIGS. 9 and 10 are conceptual diagrams showing a layout information table. In the storage unit 3000, a layout information storage unit 3005 configured by a layout information table as shown in FIG. 9 is constructed. The layout information table is information for determining how the analysis information generated by the generation AI system 4 is arranged by the terminal device 1 in the whiteboard work area 102. The layout information table associates the analysis content with the layout of the types of analysis information. In the present embodiment, as described with reference to FIG. 15, analysis information of types determined by the analysis content is generated.

[0079] For example, in the arrangement information storage unit 3005, when the analysis content is "topic and topic", an arrangement is registered in advance in which the topic is the parent hierarchy, the topic is the child hierarchy, and the topics are arranged in parallel. Similarly, the arrangement of the analysis information is determined in association with the combinations of the analysis content, "topic and summary", "topic and summary", and "topic and topic and summary".

[0080] For those with unclear correspondence relationships such as "topic and summary", it is preferable that the generation AI system 4 transmits the correspondence relationship (which topic corresponds to which summary) to the information processing device 3 together with the analysis information. The analysis information arrangement unit 36 connects a line between the topic and the summary based on the correspondence relationship.

[0081] In addition, in FIG. 9, the arrangements for the cases where the analysis content is only one of "topic", "summary", or "topic" are not registered. However, when the analysis content is only one, since there is only one type of analysis information, it is assumed that they are arranged in series or parallel, for example.

[0082] Also, as shown in FIG. 10, in the case of "divergence", an arrangement in which case 1 to case n are connected by arrows from the original data is registered in advance. The original data is the idea that the generation AI system 4 extracts from the language information. Case 1 to case n are the ideas that the generation AI system 4 develops from the original data. In the case of "convergence", an arrangement in which a summary is connected by an arrow from original data 1 to n is registered in advance. Original data 1 to n are several ideas that the generation AI system 4 extracts from the language information. The summary is the information that the generation AI system 4 summarizes original data 1 to n. As shown in FIG. 10, by connecting a line between one analysis information and the other analysis information, the generated information is displayed as a flowchart or a mind map.

[0083] The analysis information placement unit 36 refers to the placement information table based on the type of analysis information received from the generation AI system 4, and determines the placement of the analysis information. Note that the analysis information placement unit 36 determines the placement of the analysis information so as not to overlap with the existing content already displayed in the whiteboard work area 102. For example, the analysis information placement unit 36 specifies the position of the circumscribed rectangle of the existing content from the content table, determines whether the circumscribed rectangle of the analysis information to be placed overlaps, and if it overlaps, makes adjustments such as moving the analysis information or making the sub - hierarchy and grand - hierarchy two - level.

[0084] <Functional Configuration of the Terminal Device> Returning to FIG. 4, the terminal device 1 will be described. The terminal device 1 includes a first communication unit 11, a voice input unit 12, a handwriting input unit 13, a morphological analysis unit 14, a search unit 15, a first display control unit 16, a second display control unit 17, an operation reception unit 18, an update notification unit 19, and a time - series storage processing unit 21. Each of these units included in the terminal device 1 is a function or means realized by any of the components shown in FIG. 3 operating according to instructions from the CPU 501 in which a program expanded from the HD 504 onto the RAM 503. Also, the terminal device 1 has a storage unit 1000 constructed by the RAM 503 and the HD 504 shown in FIG. 3.

[0085] The first communication unit 11 communicates with the information processing device 3 via a network. The first communication unit 11 transmits voice data or handwriting data to the information processing device 3 and receives language information converted from the voice data or handwriting data. In some cases, the first communication unit 11 may also transmit language information input via a keyboard to the information processing device 3.

[0086] The voice input unit 12 receives the input of the voice collected by the microphone 518, and based on the received voice, obtains voice data as voice information. That is, the voice input unit 12 performs PCM (Pulse Code Modulation) on the voice, which is an analog signal, and converts the voice into voice data, which is digital data.

[0087] The handwriting input unit 13 accepts the input of stroke data (coordinate point sequence) by detecting the coordinates of the position where an input means such as a pen touches the touch panel. The handwriting input unit 13 may also accept the input of stroke data (coordinate point sequence) according to mouse operations. The handwriting input unit 13 generates stroke data by connecting and interpolating this coordinate point sequence. Stroke data is information displayed on the display based on the trajectory of coordinates input by the input means. The stroke data may be interpolated as appropriate. Handwriting data is data having one or more stroke data.

[0088] The morphological analysis unit 14 performs morphological analysis on arbitrary language information transmitted from the information processing apparatus 3 and decomposes it into words with part-of-speech tags. Morphological analysis means decomposing language information into the smallest meaningful units and discriminating meanings, parts of speech, etc.

[0089] The search unit 15 searches the illustration storage unit 1002, the Internet, etc. with a character string such as a word decomposed by the morphological analysis unit 14. The search unit 15 acquires an illustration that matches the character string from the illustration storage unit 1002.

[0090] The first display control unit 16 displays language information, an illustration list, menu icons, etc. in the conversation storage area 101 of the information display screen 100. The second display control unit 17 displays language information, analysis information, an illustration list, an illustration, menu icons, etc. in the whiteboard work area 102 of the information display screen 100.

[0091] The operation reception unit 18 accepts various operations in the conversation storage area 101 or the whiteboard work area 102. The operations include pressing a menu icon, selecting language information, selecting an illustration, selecting content, editing content, moving content, and deleting content, etc.

[0092] When there is a change in the content of the whiteboard working area 102, the update notification unit 19 notifies the information processing device 3 of information regarding the content in order to share it with other terminal devices 1b and 1c. The changes to the content may include changes to be shared about the content, such as position, size, color, etc.

[0093] The time-series memory processing unit 21 associates the state of the conversation accumulation area 101 with the state of the whiteboard working area 102 and stores it in the content display time-series memory unit 1001. Thereby, the content can be played back in time series while being synchronized with the language information.

[0094] (Content display time-series table) FIG. 11 is a conceptual diagram showing a content display time-series table. In the storage unit 1000, a content display time-series memory unit 1001 configured by a content display time-series table as shown in FIG. 11 is constructed. In this content display time-series table, the state of the conversation accumulation area 101 and the state of the whiteboard working area 102 are stored in association with each other in time series in association with the state ID.

[0095] · The item of the state ID is identification information of the state of the conversation accumulation area 101 and the state of the whiteboard working area 102.

[0096] · The state of the conversation accumulation area 101 indicates which language information is displayed in the conversation accumulation area 101 by the language information ID. For example, when the state ID is s001, language information t001 to t002 is displayed in the conversation accumulation area 101, and the content indicated by the state of the whiteboard working area 102 with the state ID s001 is displayed in the whiteboard working area 102 in that state. Next, when t003 is added to the conversation accumulation area 101 and language information t001 to t003 is displayed, the content indicated by the state of the whiteboard working area 102 with the state ID s002 is displayed in the whiteboard working area 102. In this way, the state of the conversation accumulation area 101 and the state of the whiteboard working area 102 are associated with each other.

[0097] · The state of the whiteboard working area 102 is indicated by the above content table. Therefore, the terminal device 1 can reproduce the state of the past whiteboard working area 102 as it is, rather than an image like a snapshot. However, the state of the whiteboard working area 102 may be stored as an image like a snapshot. Also, the entire information display screen 100 may be recorded like a video.

[0098] Note that the content display time series storage unit 1001 may be recorded by the information processing device 3 and transmitted to the terminal device 1. However, there may be a difference in the number of language information that can be displayed in the conversation storage area 101 depending on the terminal device 1. In such a case, it is preferable to save it for each terminal device 1.

[0099] (Illustration table) Regarding the illustration storage unit 1002 of the terminal device 1, its structure may be the same as that of the illustration storage unit 3003 of the information processing device 3. When the terminal device 1 executes a web app, the illustration storage unit 1002 is downloaded from the information processing device 3 to the terminal device 1 by the terminal device 1 connecting to the information processing device 3. When the terminal device 1 executes a native app, the illustration storage unit 1002 is pre-owned by the native app. The illustration storage unit 1002 may be downloaded when the terminal device 1 executes a native app.

[0100] <Settings of information display screen and whiteboard working area> Next, with reference to FIG. 12, the information display screen 100 displayed by the terminal device 1 will be described. FIG. 12 is the information display screen 100 in the initial state displayed by the terminal device 1. The information display screen 100 may be a web app or a native app.

[0101] The information display screen 100 has a conversation storage area 101, a whiteboard working area 102, a tool display bar 103, and a voice recognition result display bar 110.

[0102] The tool display column 103 displays various tools. These tools include a content selection tool 111, a pen tool 112 for handwriting input, a shape tool 113 for drawing shapes, a text tool 114 for inputting text, a lasso tool 115, etc. Also, a switching button 107 and a generative AI setting button 108 are displayed at the top. These are just examples. Also, among the tools, there is a display / hide switching tool 116 for the conversation storage area 101. By pressing the display / hide switching tool 116, the conversation storage area 101 closes and the whiteboard working area 102 expands accordingly. By pressing the switching button 107, the review mode starts, and the language information and content are reproduced based on the content display time series memory unit 1001. That is, the content is displayed in time series while synchronized with the language information. The generative AI setting button 108 will be described later.

[0103] In the conversation storage area 101, the language information converted from the voice data is displayed in time series. When new language information is added to the conversation storage area 101, the old language information moves from top to bottom. The language information that reaches the bottom of the conversation storage area 101 scrolls out. Even after scrolling out, the participants can display the past language information by operating the scroll bar.

[0104] In the voice recognition result display column 110, the language information converted from the voice data acquired by the voice input unit 12 of the terminal device 1a is displayed in real time. The language information being displayed in the voice recognition result display column 110 may also be shared with other terminal devices 1b, 1c. For the language information being displayed in the voice recognition result display column 110, illustrations are automatically displayed. For example, when the silent state continues for a certain period of time or more, the language information being displayed in the voice recognition result display column 110 is sent to the information processing device 3. The information processing device 3 sends the converted language information to each terminal device 1 indicating that it will be displayed in the conversation storage area 101, so that the language information is displayed in the conversation storage area 101 of each terminal device 1 and the language information is shared among the terminal devices 1.

[0105] Participants can freely input content in the whiteboard working area 102. That is, participants can use the whiteboard working area 102 like an electronic blackboard. Also, as will be described later, analysis information is automatically displayed in the whiteboard working area 102. Also, Participants can manually copy the language information in the conversation storage area 101 to the whiteboard working area 102.

[0106] Participants may be able to adjust the positions and sizes of the conversation storage area 101 and the whiteboard working area 102. For example, participants can swap the left and right of the conversation storage area 101 and the whiteboard working area 102. When participants enlarge the whiteboard working area 102, the conversation storage area 101 may become smaller accordingly. The same applies when participants change the size of the conversation storage area 101. Also, the whiteboard working area 102 may be wider than the display 506. In this case, the display 506 displays only a part of the whiteboard working area according to the operation of the participant.

[0107] FIG. 13 shows an example of a first setting screen 250 and a second setting screen 260 that are displayed when the generate AI setting button 108 is pressed. As shown in FIG. 13, the first setting screen 250 and the second setting screen 260 are displayed in a pop-up manner with overlap, but they may be displayed in any way, such as on separate pages. The screen information (Web app) of the first setting screen 250 and the second setting screen 260 is transmitted from the information processing device 3 to the terminal device 1, and the terminal device 1 displays it. When a native app operates on the terminal device 1, the configurations of the first setting screen 250 and the second setting screen 260 are pre-held by the native app.

[0108] <<First Setting Screen>> FIG. 14 shows an enlarged example of the first setting screen 250. The first setting screen 250 is a screen for a participant to specify an arbitrary range in the conversation storage area 101 or to specify where to place the analysis information in the whiteboard work area 102. The first setting screen 250 has a start column 252, an end column 253, a PREV SUMMARY column 254, an X column 255, and a Y column 256. The start column 252 is a column for specifying the date and time when the generation AI system 4 starts analyzing the language information. The end column 253 is a column for specifying the date and time when the generation AI system 4 ends analyzing the language information. The generation AI system 4 analyzes the language information from the start column 252 to the end column 253 to generate analysis information. As shown in FIG. 5, the input time is associated with the language information. For example, a participant can specify a range of past times to display the analysis information. Of course, a participant may also specify a range of future times, such as the scheduled time of a meeting. The start column 252 and the end column 253 are an example of a method for specifying the range of language information.

[0109] If there is already language information in the time range from the start column 252 to the end column 253, the PREV SUMMARY column 254 is a column in which a summary of the language information is displayed. The X column 255 and the Y column 256 are columns for a participant to set the display position (starting point) of the analysis information in the whiteboard work area 102.

[0110] Note that the setting of the first setting screen 250 is not essential, and only the second setting screen 260 described below may be set.

[0111] <<Second Setting Screen>> FIG. 15 shows an enlarged example of the second setting screen 260. The second setting screen 260 is a screen for a participant to set the analysis content that the information processing device 3 requests the generation AI system 4 to perform, and a screen for a participant to set the range of the language information to be analyzed. There are two methods for setting the analysis content: a method of selecting an instruction in the analysis content selection column 261 and a method of giving an instruction by inputting natural language or the like in the instruction input column 263. First, these will be described.

[0112] In the analysis content selection column 261, the participant can select the analysis content and the range of the language information in the conversation accumulation area 101 to be analyzed. First, there are five types of analysis content: summary 266, topic 267, issue 268, divergence 269, convergence 271, and keyword 272. These five types of analysis content represent the types of analysis information. The participant can select one or more of the five types simultaneously. When the summary 266 is selected, the generation AI system 4 generates a summary of the language information in the specified range of the conversation accumulation area 101. A summary refers to information that concisely summarizes the key points of the language information. When the topic 267 is selected, the generation AI system 4 generates a topic of the language information in the specified range of the conversation accumulation area 101. A topic is the subject or title of the language information expressed in a shorter text than the summary. When the issue 268 is selected, the generation AI system 4 generates an issue of the language information in the specified range of the conversation accumulation area 101. An issue refers to the theme or purpose being discussed in the meeting. When the divergence 269 is selected, the generation AI system 4 generates divergence information by diverging the content of the language information in the specified range of the conversation accumulation area 101. Divergence means coming up with a lot of various candidate ideas. When the convergence 271 is selected, the generation AI system 4 generates convergence information by converging the content of the language information in the specified range of the conversation accumulation area 101. Convergence means selecting ideas with a lot of agreement, ideas discussed for a long time, or emphasized ideas from the language information in the conversation accumulation area 101. When the keyword 272 is selected, the generation AI system 4 extracts keywords (such as nouns) from the language information in the specified range of the conversation accumulation area 101.

[0113] Note that the analysis content in FIG. 15 is an example, and it may also be possible to select search, conclusion, decision items, homework, next meeting content, etc. Search refers to among the keywords 272, buzzwords, those with a high search frequency on the network, abbreviations, those with a high speech frequency, etc. By narrowing down the search, it is possible to suppress the complexity of the whiteboard work area 102. Also, by having the generation AI system 4 generate the conclusion, decision items, homework, and next meeting content, it becomes easier for the participants to share information.

[0114] The range of the language information in the conversation storage area 101 to be analyzed will be described. In the analysis content selection column 261, there are options of time interval 273, number of characters 275, and automatic determination 277 as the generation timing of the analysis information. The time interval 273 is a setting in which, for example, the generation AI system 4 analyzes the language information within N minutes set by the participant in the time setting column 274. The number of characters 275 is a setting in which, for example, the generation AI system 4 analyzes the language information in the number set by the participant in the number of characters setting column 276.

[0115] The automatic determination 277 means that the generation AI system 4 automatically determines the range of the language information in the conversation storage area 101 to be analyzed. When the automatic determination 277 is selected, the participant can select the method for determining the generation timing. The options for the generation timing are, for example, speaker 278, change of topic 279, and keyword detection 280. The speaker 278 is a setting in which the generation timing is set to when the speaker in the conversation storage area 101 changes. That is, the specified range is the language information from when the speaker of the language information last changed until the next speaker change. The change of topic 279 is a setting in which the generation timing is set to when the topic of the language information in the conversation storage area 101 changes. That is, the specified range is the language information from when the topic last changed until the next topic change. The generation AI system 4 determines the change of topic. The keyword detection 280 is a setting in which the generation timing is set to when the keyword set in the keyword column 281 is detected from the language information in the conversation storage area 101.

[0116] The time interval 273, the number of characters 275, or the automatic determination 277 is an example of the method for specifying the range of the language information. Even if the time interval 273, the number of characters 275, or the automatic determination 277 is selected, it is also possible for the participant to manually specify the range of the language information and cause the generation AI system 4 to generate the analysis information.

[0117] In addition, when the start column 252 and the end column 253 are set on the first setting screen 250, the time interval 273, the number of characters 275, or the automatic determination 277 may be valid only within the time range set by the start column 252 and the end column 253, or may be valid regardless of the time range set by the start column 252 and the end column 253.

[0118] Next, the instruction input field 263 will be described. In the instruction input field 263, the participant can input the analysis content and the range of the language information of the conversation storage area 101 to be analyzed in natural language. The participant can turn on / off whether to give the instruction of the instruction input field 263 to the generation AI system 4 using the radio button in the instruction input field 263. The input may be by voice or keyboard input. In the instruction input field 263, the participant can input in natural language an instruction that cannot be selected in the analysis content selection field 261. For example, the participant can input "extract only questions" or "list up only the decision items" in the instruction input field 263. Also, the participant may input the same instruction as in the analysis content selection field 261 in natural language. For example, the participant can input "generate a summary every 3 minutes and automatically generate a topic when there is a change in the topic" in the instruction input field 263.

[0119] Next, the illustration setting field 262 will be described. The illustration setting field 262 allows the participant to set whether to automatically display an illustration in the whiteboard work area 102. When the radio button in the illustration setting field 262 is selected, an illustration related to the analysis information arranged in the whiteboard work area 102 is automatically displayed. Note that in the conversation storage area 101, an illustration list matching the keyword is automatically displayed regardless of the setting of the illustration setting field 262. However, the participant may be able to set whether to automatically display the illustration list in the conversation storage area 101.

[0120] In addition, the second setting screen 260 has a proposal display column 264. The proposal display column 264 proposes to the participants what to select in the analysis content selection column 261 by the generation AI system 4, or proposes what instructions to enter in the instruction input column 263. For example, in FIG. 15, a proposal such as "Since it is still the beginning of the meeting, how about diverging the ideas that are currently out?" is displayed. For example, at the end of the meeting, it is expected that the generation AI system 4 will propose something like "Since it is the end of the meeting, how about converging the ideas so far?" The participant can turn on / off whether to receive the proposal from the generation AI system 4 with the radio button in the proposal display column 264.

[0121] The first setting screen 250 or the second setting screen 260 can be displayed on any terminal device 1. If the analysis content arbitrarily set on each terminal device 1 is valid in the information processing device 3, analysis information corresponding to the analysis content (which may be the same or different) generated from the same language information can be displayed in the whiteboard work area 102. For example, a case may occur where a plurality of topics are generated from the same range of language information (since the range of language information is the same, a plurality of the same topics are displayed). In order to suppress the display of such complicated analysis information, the following exclusive control may be performed. · The information processing device 3 generates analysis information only once from the same language information. Since the language information from which the analysis information is generated is registered in the content table, the analysis information acquisition unit 37 does not transmit the language information registered in the content table to the generation AI system 4. In this case, the information processing device 3 may transmit an error message to the terminal device 1 that has transmitted the analysis content. · The information processing device 3 restricts the terminal device 1 that can display the first setting screen 250 or the second setting screen 260 to, for example, the organizer of the meeting. The information processing device 3 transmits the authority to display only the first setting screen 250 or the second setting screen 260 to the terminal device 1 of the organizer. When the generation AI setting button 108 is pressed on a terminal device 1 without authority, an error message is displayed.

[0122] <Voice Recognition and Sharing of Language Information> Next, referring to FIG. 16, the process in which each terminal device 1 participating in the meeting shares the language information converted from the voice data will be described. First, the case where the generation AI system 4 does not generate analysis information will be described. FIG. 16 is a sequence diagram illustrating the process in which the terminal device 1a transmits voice data to the information processing device 3, and the information processing device 3 shares the language information converted from the voice data with other terminal devices 1b and 1c. The terminal device 1 and other terminal devices 1a and 1b participate in the same meeting by connecting to the URL invited by email or the like, and the information processing device 3 manages the plurality of terminal devices 1 participating in the same meeting by participant ID, IP address, or the like. Alternatively, each participant may select the meeting registered in the meeting management table on the terminal devices 1a to 1c and enter a predetermined password to participate in the same meeting. Also in this case, the information processing device 3 manages the plurality of terminal devices 1 participating in the same meeting by participant ID, IP address, or the like.

[0123] S101: The voice input unit 12 of the terminal device 1a receives the input of the voice spoken by the participant.

[0124] S102: The voice input unit 12 transmits the voice data converted into digital data to the information processing device 3 via the first communication unit 11.

[0125] S103: The second communication unit 31 of the information processing device 3 receives the voice data, and the conversion unit 32 performs voice recognition on the voice data and converts it into language information based on the voice data.

[0126] S104,S105: The second communication unit 31 transmits the language information to other terminal devices 1b and 1c participating in the meeting. Note that since the voice recognition is performed in real time, the content of the speech may not be determined at this point.

[0127] S106,S107: The first communication units 11 of the terminal device 1a and other terminal devices 1b and 1c receive the language information, and the first display control unit 16 displays the undetermined language information in the voice recognition result display column 110 at any time.

[0128] S108: When the conversation content management unit 33 detects the break of a speech based on, for example, a silent period, it saves in the conversation content table the language information ID (newly numbered), language information, input time, speaker (the logged-in participant), and like count (initial value is zero).

[0129] S109, S110: The second communication unit 31 transmits the language information ID, language information, and input time to the terminal device 1 and other terminal devices 1b, 1c.

[0130] S111, S112: The first communication unit 11 of the terminal device 1a and other terminal devices 1b, 1c receives the language information ID, language information, and input time, and the first display control unit 16 adds and displays the new language information in the conversation storage area 101. The input time may be absent, but the input time is used for highlighting the language information corresponding to the content selected in the whiteboard work area 102.

[0131] Also, since the state of the conversation storage area 101 has changed, the time-series memory processing unit 21 currently acquires the language information ID being displayed in the conversation storage area 101 as the state of the conversation storage area 101. Also, the time-series memory processing unit 21 acquires the content data being displayed in the whiteboard work area 102 as the state of the whiteboard work area 102 (it may be acquired from the information processing device 3 or from the content being displayed on the terminal device 1). The time-series memory processing unit 21 associates the state of the conversation storage area 101 and the state of the whiteboard work area 102 and saves them in the content display time-series memory unit 1001.

[0132] <<Generation of Analysis Information by the Generation AI System>> Next, with reference to FIG. 17, the flow in which each terminal device 1 participating in the meeting displays the analysis information generated by the generation AI system 4 by analyzing the language information will be described. FIG. 17 is a sequence diagram illustrating the process in which the terminal device 1 displays the analysis information generated by the generation AI system 4 by analyzing the language information in the whiteboard work area 102.

[0133] S121: The participant operates the terminal device 1a to set the analysis content and the like on the second setting screen 260. The operation reception unit 18 of the terminal device 1a receives the participant's setting. The setting on the second setting screen 260 can be done before the start of the meeting or arbitrarily during the meeting. If the date and time range and the like are set on the first setting screen 250, the setting content is also transmitted to the information processing device 3. The analysis information is displayed at the initial position of the analysis information set on the first setting screen 250.

[0134] S122: The first communication unit 11 transmits the analysis content to the information processing device 3.

[0135] S123: The second communication unit 31 of the information processing device 3 receives the analysis content, and the third communication unit 38 transmits it to the generation AI system 4. In this way, the analysis content only needs to be transmitted when the participant changes it. However, the analysis content may be transmitted each time together with the language information.

[0136] After this, each participant participating in the meeting makes a speech, and steps S102 to S112 in FIG. 16 are repeatedly executed.

[0137] S124: When the language in the range set by the analysis content is accumulated, the analysis information acquisition unit 37 of the information processing device 3 transmits the language information (including the speaker and date and time of each language information) to the generation AI system 4. The language in the range set by the analysis content is determined by a time interval, the number of characters, or automatic determination (change of speaker, topic). In the case of keyword extraction, the language information is transmitted to the information processing device 3 without being accumulated. The analysis information acquisition unit 37 detects a time interval, the number of characters, a change of speaker, and a change of topic, and summarizes and transmits the language information in the specified range to the generation AI system 4.

[0138] Also, when a time interval, the number of characters, or automatic determination is set, the language information may be transmitted to the generation AI system 4 without being accumulated. This is because the generation AI system 4 can determine a time interval, the number of characters, a speaker, and a change of topic. When the language information is not accumulated, following step S107 in FIG. 16, the language information is transmitted to the generation AI system 4 each time.

[0139] S125: The AI generation system 4 analyzes language information and the like based on the analysis content, generates analysis information, and returns it to the information processing device 3. The type of each analysis information is also attached to the analysis information.

[0140] S126: The analysis information acquisition unit 37 of the information processing device 3 acquires the analysis information via the third communication unit 38.

[0141] S127: The analysis information arrangement unit 36 refers to the arrangement information table and determines the arrangement of the analysis information so as not to overlap with the existing content. Also, the analysis information arrangement unit 36 registers information related to the analysis information in the content table.

[0142] S128, S129: The second communication unit 31 of the information processing device 3 transmits the analysis information and the position information to the terminal devices 1a to 1c. This position information may be the coordinates (such as the upper left corner and size of the message box) of each analysis information determined by the analysis information arrangement unit 36.

[0143] S130, S131: The first communication unit 11 of the terminal devices 1a to 1c receives the analysis information and the position information, and the second display control unit 17 causes the analysis information to be displayed in the whiteboard work area 102 based on the position information.

[0144] As described above, as shown in FIG. 1, the topic and the topic are displayed in the whiteboard work area 102.

[0145] <Example of display of analysis information> FIG. 18 shows an example of the arrangement of analysis information displayed on the terminal device 1. In FIG. 18, a message box including the issue 211 and topics 212 to 214 shown in FIG. 1 is displayed. Further, a message box including another issue 301 and topics 302 to 306 is also displayed. A set of an issue and topics (for example, issue 211 and topics 212 to 214, issue 301 and topics 302 to 306) is generated from language information within a range specified by a time interval, the number of characters, or automatic determination. One message box has only one type of analysis information, but there may be cases where there are multiple pieces of analysis information even if there is only one type of analysis information set on the second setting screen 260 (topics or summaries).

[0146] Analysis information of the same type generated from the specified range of language information may be grouped. For example, the second display control unit 17 displays a dotted line frame 307 or the like that surrounds analysis information of the same type as the analysis information over which the mouse pointer 308 is hovered, making it easier for the participants to grasp the analysis information of the same group. Such grouping can be easily realized because the type of each piece of analysis information is transmitted from the information processing device 3 to the terminal device 1.

[0147] Also, in FIG. 18, there is no connection line indicating a hierarchical structure between one piece of analysis information and the other piece of analysis information, but as shown in FIGS. 9 and 10, the analysis information may be connected to each other. The participant may be able to set on the terminal device 1 whether to connect or not. Also, since the arrangement such as issue 211 and topics 212 to 214 is the arrangement determined by the analysis information arrangement unit 36, the participant can move arbitrarily in units of message boxes.

[0148] Also, the message box does not necessarily have to be visible, and the analysis information may be arranged in a transparent message box. Also, the participant can set the font, font size, color, etc. of the analysis information within the message box.

[0149] In this way, as the meeting progresses, the analysis information (and other content as well) is placed in the whiteboard working area 102. Since overlapping with past content would hinder the participants' understanding of the information, the analysis information placement unit 36 controls the placement of the analysis information so as not to overlap based on the coordinates and sizes of the existing content (recorded in the content table). Reduced content 309 is displayed on the right side of the whiteboard working area 102 in FIG. 18. The participants can arbitrarily reduce and enlarge the content of the meeting that has ended. The analysis information placement unit 36 can also control the placement of the analysis information so as not to overlap with such reduced content 309.

[0150] Also, as shown in FIG. 19, the area 310 where the analysis information is to be placed may be specifiable by the participants. FIG. 19 is an example of the area 310 where the analysis information is to be placed. In FIG. 19, the display position of the analysis information is set as the area 310. This can prevent the content in the whiteboard working area 102 from becoming complicated. Note that the setting of the area 310 can be performed by the participants using a pointing device such as a mouse.

[0151] Even when the area 310 is set, the analysis information placement unit 36 of the information processing device 3 can perform the placement of the analysis information. When the area 310 is set, the second display control unit 17 of the terminal device 1 converts the position information transmitted from the information processing device 3 into coordinates within the area 310 and displays the analysis information. For example, the second display control unit 17 determines the position of the upper left corner of the new analysis information so as not to overlap with the analysis information already displayed in the area 310. Then, the second display control unit 17 determines whether the new analysis information can fit into the area 310 when placed at the determined position of the upper left corner. When the analysis information protrudes, for example, in the width direction, the number of stages of the placement information is increased to adjust so that it does not protrude. By doing so, the second display control unit 17 can place the analysis information in the area 310 while maintaining the relative positions between the analysis information (such as the relative positions of the topics and sub - topics) as much as possible.

[0152] Further, the terminal device 1 may transmit the coordinates of the area 310 to the information processing device 3. The analysis information arrangement unit 36 of the information processing device 3 identifies the content in the area 310 using the content table, and determines the arrangement of new analysis information so that the new analysis information fits within the area 310.

[0153] Also, the second display control unit 17 preferably displays the analysis information in a display mode corresponding to the type of analysis information or the like. The display mode refers to, for example, boldface, emphasized color, large character size, surrounding the analysis information with a graphic, or changing the shape of the surrounding graphic. Such a display mode is also referred to as highlighted display. For example, in FIG. 19, the backgrounds of the topics 211 and 301 are colored. This makes it easier for the participants to notice the topics 211 and 301 with high importance. In addition, the second display control unit 17 may change the color of the text or the background for each type of analysis information. This allows the participants to determine the type of analysis information by color.

[0154] Also, in FIG. 19, the name of the topic is underlined. In this way, the second display control unit 17 can highlight the analysis information in units of character strings. In addition, the second display control unit 17 may highlight names and frequently appearing keywords with underlines, boldface, emphasized colors, etc. The detection of names and keywords may be performed by any of the information processing device 3, the terminal device 1, or the generation AI system 4. When the information processing device 3 performs the detection, the information processing device 3 designates the type of analysis information for each analysis information, and further designates proper nouns (such as names) and frequently appearing keywords. The second display control unit 17 of the terminal device 1 changes the color of the analysis information according to the type of analysis information, and also underlines or displays proper nouns in a predetermined color.

[0155] <<Example of display of illustration based on analysis information>> Next, referring to FIG. 20, an illustration displayed based on the analysis information will be described. FIG. 20 shows an example illustration displayed together with the analysis information in the whiteboard work area 102. When the radio button in the illustration setting field 262 is selected on the second setting screen 260, an illustration related to the analysis information arranged in the whiteboard work area 102 is automatically displayed. In FIG. 20, an illustration 215 generated from topic 212 is displayed under topic 212, an illustration 216 generated from topic 213 is displayed under topic 213, and an illustration 217 generated from topic 214 is displayed under topic 214. Note that an illustration may be similarly displayed for issue 211 as well.

[0156] A method of displaying such illustrations 215 to 217 will be described. The morphological analysis unit 14 decomposes the analysis information (in this case, the topic) character by character, only with Chinese characters, or into a character string by morphological analysis, and the search unit 15 searches the illustration storage unit 1002. The second display control unit 17 displays the illustrations that match the search around (below in the figure) topics 212 to 214. The display position of the illustrations is determined so as not to overlap with other contents. A part of illustrations 215 to 217 may overlap with topics 212 to 214 from which they are derived.

[0157] The method of obtaining illustrations is not limited to searching the illustration storage unit 1002. For example, the search unit 15 may obtain illustrations by the following methods. · Method 1 is a method of image searching for illustrations with a high similarity to the illustrations that match the search in the illustration storage unit 1002 or on the Internet (search engine). · Method 2 is a method of searching for images on the Internet with a character string obtained by morphological analysis. · Method 3 is a method of transmitting a character string obtained by morphological analysis to the generation AI system 4 and having an image generated. Alternatively, the information processing device 3 obtains an illustration together with the analysis information from the generation AI system 4.

[0158] By displaying Illustrations 215 to 217, participants can more easily understand the content of the language information than when only the issues and topics are arranged. Also, instead of displaying only one illustration 215 to 217 for each of the topics 212 to 214 as shown in FIG. 20, the second display control unit 17 may display an illustration list (see FIG. 22) first. Participants can select a desired illustration from the illustration list.

[0159] <<Determination of Arrangement of Analysis Information Using Machine Learning>> In the arrangement information tables shown in FIGS. 9 and 10, the arrangement of the analysis information is determined by the analysis content, but the content and arrangement of each analysis information may be modeled by machine learning. The model is modeled, for example, by deep learning that takes as input the number for each type of analysis information and outputs the identification information of the arrangement information (the arrangement number in the arrangement information table) as the correct answer. This model outputs the probability that the pre-registered arrangement information as shown in FIGS. 9 and 10 is selected for the input of the number for each type of analysis information.

[0160] FIG. 21 is a functional block diagram of an arrangement information output unit 220 that uses machine learning. As shown in FIG. 21, the arrangement information output unit 220 can include a number acquisition unit 221, a learning data storage unit 222, a machine learning unit 223, a learned model storage unit 224, and an inference unit 225. Each will be described below.

[0161] <Learning Phase> The number acquisition unit 221 acquires the number for each type of analysis information generated by the generation AI system 4. For example, when there is 1 issue and 3 topics, the number acquisition unit 221 acquires "issue: 1, topic: 3, summary: 0, divergence: 0, convergence: 0". The learning data storage unit 222 stores learning data for machine learning. The learning data in the learning data storage unit 222 is the number for each type of analysis information (input) acquired and accumulated by the number acquisition unit 221 over a certain period and the identification information of the corresponding arrangement information (output).

[0162] The machine learning unit 223 generates a trained model that has learned the correspondence between the number (input) for each type of analysis information and the identification information (output) of the corresponding arrangement information. Further, the machine learning unit 223 stores the generated trained model in the trained model storage unit 224. The trained model storage unit 224 stores the trained model generated by the machine learning unit 223.

[0163] <Inference phase> The inference unit 225 obtains, from the number acquisition unit 221, the number for each type of analysis information generated by the generation AI system 4 and infers the identification information of the arrangement information. Specifically, the inference unit 225 obtains, from the number acquisition unit 221, the number for each type of analysis information. Further, the inference unit 225 inputs the number for each type of analysis information to the trained model in the trained model storage unit 224 to output the identification information of the arrangement information.

[0164] Further, instead of the information processing device 3 determining the arrangement information, the arrangement information may also be obtained from the generation AI system 4 together with the analysis information.

[0165] <Display of illustration in conversation accumulation area> Subsequently, with reference to FIG. 22, a method for displaying an illustration in the conversation accumulation area will be described. FIG. 22 is a diagram schematically illustrating a process in which the terminal device 1 displays an illustration based on the language information in the conversation accumulation area 101. Further, in FIG. 22, an operation in which the participant manually displays the language information displayed in the conversation accumulation area 101 in the whiteboard work area 102 without performing analysis by the generation AI system 4 will be described.

[0166] FIG. 22(a) shows the language information displayed in the conversation storage area 101. In FIG. 22(a), there is no content yet in the whiteboard work area 102, but there may be. The participant selects (taps or clicks) any language information 121 in the conversation storage area 101 for use in the whiteboard work area 102. The selected language information 121 is decomposed character by character, only Chinese characters, or into a character string by morphological analysis, and the search unit 15 searches the illustration storage unit 1002. In addition, as a method for obtaining illustrations, the same method as in FIG. 20 can be adopted.

[0167] As a result, as shown in FIG. 22(b), the first display control unit 16 displays an illustration list 122 related to the selected language information 121 in the conversation storage area 101 around it. The illustration list 122 is a thumbnail of a plurality of illustrations (there may be one illustration).

[0168] The participant can select an illustration to be displayed in the whiteboard work area 102 from among the illustration list 122. The participant may drag and drop the selected illustration into the whiteboard work area 102, or simply select the illustration in the illustration list 122, and the illustration may be displayed in the whiteboard work area 102.

[0169] The selection of an illustration from among the illustration list 122 can be made not only for one's own speech content but also for the speech of others. As a result, others will feel that their own speech has been used, and a sense of active participation can be obtained. In FIG. 22(b), it is assumed that the participant did not select an illustration.

[0170] In addition, the terminal device 1 may hold an illustration displayed for the language information displayed in the speech recognition result display column 110 and display it as the illustration list 122. The time from when the participant selects the language information 121 until the illustration list 122 is displayed can be shortened.

[0171] As shown in FIG. 22(c), the participant selected a partial string 123 of the language information, and dragged and dropped a copy of the string 123 (hereinafter referred to as string 126) from the conversation storage area 101 to the whiteboard work area 102. FIG. 22(d) shows the string 126 displayed in the whiteboard work area 102. Therefore, the participant can understand how their speech has affected the whiteboard work area 102.

[0172] Note that the participant can also display the string 126 by clicking on a part or the whole of the string 123 and then clicking in the whiteboard work area 102. Regarding the illustrations in the illustration list 122, they can be displayed in the whiteboard work area 102 either by drag-and-drop or clicking.

[0173] In this way, the participant can reduce the input work to the whiteboard work area 102 by using the language information in the conversation storage area 101. The participant can further move or rotate the string 126 at an arbitrary angle. For the string 126 moved to the whiteboard work area 102, the search unit 15 may automatically search the illustration storage unit 1002 etc., and the second display control unit 17 may display the illustration 127. Alternatively, the participant may manually display the illustration 127.

[0174] Note that as shown in FIG. 22(a), a like button 124 and a favorite button 125 are displayed for each language information in the conversation storage area 101. The like button 124 and the favorite button 125 may be always displayed, or may be displayed when the participant selects or hovers over the language information. When the participant presses the like button 124, the language information ID and the fact that the like button has been pressed are transmitted to the information processing device 3, and the like count in the conversation content table increases by one. Also, when the participant presses the favorite button 125, the language information ID is recorded in association with this participant ID.

[0175] Also, as shown in FIG. 22(d), when content is added to the whiteboard work area 102, the update notification unit 19 transmits the content type, page, coordinates, size, font size, and language information ID to the information processing device 3 via the first communication unit 11. Since the content management unit 34 transmits a record of the newly added content table to the other terminal devices 1b and 1c, the other terminal devices 1b and 1c can share the state of the whiteboard work area 102.

[0176] Also, when the illustration list 122 as shown in FIG. 22 is displayed, the first display control unit 16 may determine the display order of the thumbnails in the illustration list 122 according to the original language information 121.

[0177] FIG. 23 is a diagram for explaining the display order of the thumbnails of the illustrations determined according to the language information. FIGS. 23(a) and (b) show how the display order of the thumbnails changes according to different language information.

[0178] The first display control unit 16 performs morphological analysis on the language information and searches the illustration table in order from the first word among the plurality of words obtained by the morphological analysis. The first display control unit 16 extracts the illustrations that match the words and displays them as individual thumbnails in the illustration list. At this time, even if a plurality of illustrations match a single word in the illustration table, if the first display control unit 16 arranges all of them as thumbnails in the illustration list, the display area of the finite illustration list will be filled only with the thumbnails of the illustrations related to a single word. To prevent this, the first display control unit 16 displays only one thumbnail per word for one sentence (an example of language information in a predetermined unit) of the language information. After the first display control unit 16 finishes displaying the thumbnails of the illustrations for one sentence, it returns to the beginning of the same sentence and displays the continuation of the thumbnails.

[0179] Figure 23(a) shows the illustration list 230 displayed for the language information 239 of "people who like animals" spoken by the participant. The words detected by morphological analysis are, in order, "animal", "like", and "person". Since the first display control unit 16 displays only one thumbnail per word, in Figure 23(a), the thumbnails 231 corresponding to "animal", 232 corresponding to "like", and 233 corresponding to "person" are displayed in this order. Also, when the first display control unit 16 finishes displaying the thumbnails of illustrations for one sentence, it returns to the beginning of the same sentence and displays the continuation of the thumbnails. For this reason, in Figure 23(a), subsequently, the thumbnails 234 corresponding to "animal", 235 corresponding to "like", 236 corresponding to "person", 237 corresponding to "animal", and 238 corresponding to "like" are displayed in this order.

[0180] Figure 23(b) shows the illustration list 240 displayed for the language information 249 of "likes people and animals" spoken by the participant. The words detected by morphological analysis are, in order, "person", "animal", and "like". The order of appearance of the words is different from that in Figure 23(a). For this reason, in Figure 23(b), the thumbnails 241 corresponding to "person", 242 corresponding to "animal", 243 corresponding to "like", 244 corresponding to "person", 245 corresponding to "animal", 246 corresponding to "like", 247 corresponding to "person", and 248 corresponding to "animal" are displayed in this order.

[0181] Such display control of thumbnails in the illustration list is similarly possible in the whiteboard work area 102.

[0182] Note that the first display control unit 16 may display a plurality of thumbnails per word as long as the area is not filled only with thumbnails of illustrations related to one word. Also, the first display control unit 16 may perform morphological analysis collectively on a plurality of sentences and display only the illustrations common to the words detected as a result.

[0183] <Utilization of the temporary area> Although the content in the whiteboard working area 102 is what the participants use as the deliverable, since the analysis information generated by the generative AI system 4 is automatically displayed, the participants do not necessarily want to display all the analysis information in the whiteboard working area 102. The content in the whiteboard working area 102 is likely to increase and thus become cumbersome. Since the whiteboard working area 102 is used by the participants as a whiteboard, there is also an idea that only the content input by the participants with intention should be placed.

[0184] Therefore, referring to FIG. 24, a use case using a temporary area 330 (an example of a third area) will be described. FIG. 24 shows an information display screen 100 including the temporary area 330 that the terminal device 1 displays during a meeting. The analysis information transmitted from the information processing device 3 is collectively displayed in the temporary area 330. The method for determining the arrangement of the analysis information in this case may be the same as when the area 310 is set.

[0185] The participants can check the analysis information in the temporary area 330 and drag and drop the analysis information into the whiteboard working area 102 with a mouse or the like. In this way, the participants can easily move the analysis information from the temporary area 330 to the whiteboard working area 102. Note that the analysis information in the temporary area 330 may be deleted (moved) or remain as it is (copied).

[0186] In FIG. 24, topics 212 to 214 and topics 302 and 306 are copied from the temporary area 330 to the whiteboard working area 102. Since it is troublesome for the participants to copy each piece of analysis information, for example, it is preferable that the information can be moved for each type of analysis information. For example, the second display control unit 17 displays a dotted line frame 311 or the like that surrounds the analysis information of the same type as the analysis information moused over by the mouse pointer 308, and the participants can move the entire dotted line frame 311 to the whiteboard working area 102.

[0187] In FIG. 24, language information 312, 314, illustrations 313, 315 are displayed in the whiteboard work area 102. This language information 312, 314 may be input by the participants or may be copied from the conversation storage area 101. The illustrations 313, 315 are displayed by searching, for example, the illustration storage unit 1002 with the language information 312, 314.

[0188] In FIG. 24, as described in FIG. 22, illustration lists 321 to 325 are displayed in the language information of the conversation storage area 101. Such illustration lists 321 to 325 can also be displayed for the analysis information in the temporary area 330, as shown in FIG. 25.

[0189] FIG. 25 shows an information display screen 100 in which an illustration list 341 is displayed in the temporary area 330. When the participant designates arbitrary analysis information, the search unit 15 searches, for example, the illustration storage unit 1002 with the analysis information, and the second display control unit 17 displays the illustration list 341. The display method of the illustration list 341 may be the same as that of the conversation storage area 101. The participant can drag and drop an arbitrary illustration in the illustration list 341 to the whiteboard work area 102.

[0190] Note that the summary image 342 in the temporary area 330 is an image generated by the generation AI system 4 when a summary is set for the analysis content. Thus, there are cases where the analysis information can be directly obtained as an image. The participant can drag and drop the summary image 342 to the whiteboard work area 102.

[0191] <<Transfer process from the temporary area to the whiteboard work area>> FIG. 26 is a sequence diagram for explaining the process in which the terminal device 1 copies analysis information from the temporary area 330 to the whiteboard work area 102 in response to an operation by the participant.

[0192] S141: The participant drags and drops the analysis information in the temporary area 330 to the whiteboard work area 102.

[0193] S142: The operation reception unit 18 receives an operation of dragging and dropping the analysis information in the temporary area 330 to the whiteboard work area 102.

[0194] S143: The update notification unit 19 detects that the state of the whiteboard work area 102 has changed. The update notification unit 19 transmits a content update notification (temporary content ID) to the information processing apparatus 3 via the first communication unit 11.

[0195] Note that since the analysis information is initially displayed in the temporary area 330, it is registered in the content table for the temporary area 330. This content table may be the same as that in FIG. 6. The temporary content ID is the content ID in the content table for the temporary area 330.

[0196] S144: The second communication unit 31 of the information processing apparatus 3 receives the content update notification, and the content management unit 34 assigns a content ID. The content management unit 34 registers the information regarding the analysis information specified by the temporary content ID in the content table together with the content ID. Also, when the analysis information is deleted from the temporary area 330, the content management unit 34 deletes the information regarding the deleted analysis information from the content table for the temporary area 330.

[0197] S145: The second communication unit 31 of the information processing apparatus 3 transmits the new content ID to the terminal device 1.

[0198] S146: Also, since a change has occurred in the content table, the content management unit 34 transmits a content sharing request (content ID, content type, analysis information, page, coordinates, size, font size, language information ID) to other terminal devices 1b and 1c participating in the same meeting.

[0199] S147: The first communication units 11 of the other terminal devices 1b and 1c receive the content sharing request, and the second display control unit 17 displays the analysis information at the position specified by the coordinates in the specified font size.

[0200] [Application to HMD] The display unit that displays the conversation storage area 101 and the whiteboard work area 102 of the present embodiment can be an HMD (head-mounted display).

[0201] Figure 27 shows a hardware block diagram of the HMD. As shown in Figure 27, the HMD, as a computer, includes a CPU 401, a ROM 402, a RAM 403, an electronic compass 404, a gyro sensor 405, an acceleration sensor 406, an external device connection I / F 407, a display 408, an operation unit 409, a media I / F 410, and a speaker 412. These components are connected to a bus line 415.

[0202] Among these, the CPU 401 controls the overall operation of the HMD. The ROM 402 stores programs used for driving the CPU 401 such as the IPL. The RAM 403 is used as a work area for the CPU 401.

[0203] The external device connection I / F 407 is an interface for connecting various external devices. The external devices in this case are the information processing device 3, the earphone with a microphone 6, etc.

[0204] The display 408 is a type of display unit such as a liquid crystal or an organic EL (Electro Luminescence) that displays various images.

[0205] The operation unit 409 is an input means for performing various operations such as selecting and executing various instructions, selecting a processing target, and moving a cursor, including various operation buttons, a power switch, physical buttons, and a gaze operation circuit that detects and operates on the gaze of participants.

[0206] The media I / F 410 controls the reading or writing (storage) of data to / from a recording medium 209m such as a flash memory. The recording medium 209m includes DVDs, Blu-ray Discs (registered trademarks), etc.

[0207] The speaker 412 is a circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voices.

[0208] The electronic compass 404 calculates the orientation of the HMD from the Earth's magnetism and outputs orientation information.

[0209] The gyro sensor 405 is a sensor that detects changes in angles (roll angle, pitch angle, yaw angle) associated with the movement of the HMD.

[0210] The acceleration sensor 406 is a sensor that detects acceleration in three axial directions.

[0211] The bus line 415 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401.

[0212] <Main effects> In the communication system of this embodiment, since the issues and topics during the meeting are automatically displayed in the whiteboard work area 102, the burden on the participants can be reduced, and it becomes easier to grasp the ideas and the flow of the discussion. Since the range of language information is specified, the displayed analysis information can be restricted. The participants can grasp only the useful information. Also, based on the analysis content set by the participants, the communication system extracts the ideas included in the communication and summarizes the flow of the discussion. By these means, the burden on the participants in the communication can be reduced. Also, since it is possible to display illustrations in the analysis information, the participants can more easily grasp the content and flow of the discussion.

[0213] [Second Embodiment] In this embodiment, a communication system 5 that displays the analysis results obtained by the generation AI system 4 analyzing language information in the conversation accumulation area 101 will be described.

[0214] <Functional Configuration of the Communication System> FIG. 28 is an example of a block diagram showing the functions of the communication system 5 of the present embodiment divided into blocks for each of the information processing apparatus 3 and the terminal apparatus 1. In FIG. 28, mainly the differences from FIG. 4 may be described.

[0215] First, the terminal device 1 newly has a third display control unit 22. The third display control unit 22 causes the AI analysis result to be displayed in the AI analysis area 104 described with reference to FIG. 30 and the like. The AI analysis result is information generated based on the analysis result obtained by analyzing the language information by the generation AI system 4, and is hereinafter simply referred to as the "AI analysis result". Further, the terminal device 1 does not have a search unit 15 and an illustration storage unit 1002. This is because in the present embodiment, the generation AI system 4 determines the illustration. However, even in the form in which the generation AI system 4 determines the illustration, the search unit 15 may further search the illustration storage unit 1002 so that the participant can select a favorite illustration.

[0216] Further, the terminal device 1 has an AI analysis result storage unit 1004 in the storage unit 1000. The AI analysis result storage unit 1004 stores the AI analysis result obtained by the generation AI system 4 analyzing the language information. Note that the information processing apparatus 3 may have the same as the AI analysis result storage unit 1004.

[0217] Similarly, the information processing apparatus 3 does not have an analysis information arrangement unit 36 and an arrangement information storage unit 3005. This is because in the present embodiment, the generation AI system 4 arranges the analysis information, and further, the participant arranges the content using the template.

[0218] FIG. 29 schematically shows the AI analysis result held in the AI analysis result storage unit 1004. As will be described later, in the AI analysis result storage unit 1004, one or more key sentences, illustrations, and tags are associated with one major item and stored. The number of major items is one or more, and one or more key sentences, illustrations, and tags are associated with one major item. One major item and one or more key sentences, illustrations, and tags form a set. Details will be described later.

[0219] <Overview of the Conversation Storage Area> Next, with reference to FIGS. 30 to 33, the AI analysis area 104 of the present embodiment will be described. FIG. 30 shows a part of the AI analysis area 104 of the present embodiment. The AI analysis area 104 (an example of the fourth area) is displayed at the same position as the conversation storage area 101, and the participant switches between and uses both. The participant switches which of the conversation storage area 101 and the AI analysis area 104 to display using tabs 132 and 133. When tab 132 is pressed, the conversation storage area 101 is displayed, and when tab 133 is pressed, the AI analysis area 104 is displayed. However, the participant may be able to display the conversation storage area 101 and the AI analysis area 104 separately at the same time.

[0220] In addition, the settings made by the participant for the AI analysis results displayed in the AI analysis area 104 are designed so as not to be a heavy operation burden. For example, the settings described in FIGS. 14 and 15 are not necessary in the present embodiment. For example, when the participant turns on the AI analysis function once (turns on the toggle button 131 described later), the analysis information acquisition unit 37 determines the delimiter of a specific number of characters for the AI analysis, and transmits the language information to the AI generation system 4 via the third communication unit 38. The first communication unit 11 of the terminal device 1 receives the AI analysis result via the information processing device 3, and the third display control unit 22 displays the AI analysis result in the AI analysis area 104 at any time. In this way, a detailed setting screen for the AI generation system 4 is not provided, and the settings prepared in advance by the system side are applied.

[0221] The AI analysis function is turned on and off with the toggle button 131. That is, when the participant selects AI analysis with tab 133 and turns on the toggle button 131, the AI analysis result is displayed in the AI analysis area 104. However, when the participant turns on the toggle button 131, the third display control unit 22 may automatically display the AI analysis area 104.

[0222] Note that only the participants who are attending the meeting as the administrator (facilitator) of the meeting can operate the toggle button 131. The facilitator is, for example, the person in charge of the meeting progress. The facilitator may be determined at the time of login and then fixed, or may be movable from participant to participant during the meeting. However, any participant may turn the toggle button 131 on or off. In this case, it is preferable that there are two usage methods: one where all participants in the meeting can view the same AI analysis result regardless of who turns the toggle button 131 on, and the other where only the participant who turns the toggle button 131 on can view the AI analysis result.

[0223] <<Example of display of AI analysis area>> Figure 31 is a diagram for explaining the AI analysis result displayed in the AI analysis area 104. In the AI analysis area 104, a major item 141 is displayed, and one or more key sentences 142 associated with the major item 141 are displayed. The major item 141 is the title of the language information. The major item 141 corresponds to, for example, the topic being discussed in the meeting and is information that can grasp the whole of the discussion. The key sentence 142 corresponds to the keywords judged to be important in the discussion and is information extracted from the language information or information summarizing the content of the discussion in a short form.

[0224] In Figure 31, one major item 141 is displayed, but there may be multiple major items 141. The generation AI system 4 automatically detects changes in the discussion and generates a new major item 141 when it detects a change. Until the major item 141 changes, the generation AI system 4 automatically determines the key sentences 142 judged to be important for the major item 141 and associates them with the major item 141.

[0225] Also, as shown in FIG. 31, an illustration 143 is displayed in association with the major item 141. Also, one illustration 144 is displayed in one-to-one correspondence with one key sentence 142. In FIG. 31, the illustration 144 is arranged on the right side of the key sentence 142, but the position of the illustration 144 is just an example. Also, a plurality of illustrations may be displayed in association with one key sentence 142. In the present embodiment, the illustrations 143 and 144 that the generation AI system 4 determines represent the content of the major item 141 or the key sentence 142 are displayed.

[0226] Note that it is preferable that a plurality of alternative illustrations are registered in advance in the generation AI system 4. The plurality of alternative illustrations may be registered in the generation AI system 4 by the information processing device 3, or may be specified by the participant for the illustrations held by the generation AI system 4. Thereby, the range of selection of illustrations by the generation AI system 4 is suppressed, and the same illustration is likely to be always displayed for major items or key sentences with similar content.

[0227] The display and non-display of the key sentence 142 can be switched by the participant for each major item 141 by operating the open / close button 145. It can be used, for example, to display only the key sentence 142 of the major item 141 that the participant is focusing on. Even if the AI analysis area 104 is narrow, only the major item 141 that the participant is focusing on and its key sentence 142 can be displayed.

[0228] Also, one major item 141 has one arrangement button 146. The arrangement button 146 is a button for displaying the major item 141 and all the key sentences 142 associated with the major item 141 in the whiteboard work area 102. Details will be described with reference to FIG. 34. However, also in the present embodiment, the participant can display the major item and the key sentence in the whiteboard work area 102 by drag & drop or the like.

[0229] Figure 32 shows the AI analysis area 104 where a plurality of major items 141 are displayed. In Figure 32, five major items 141 are displayed. The sixth and subsequent major items 141 can be displayed by the participant scrolling the AI analysis area 104 vertically. The number of major items 141 displayed during the meeting can vary depending on the content of the language information and whether the generation AI system 4 determines that the content of the discussion has changed. Also, major items 141 are added as needed according to the progress of the discussion. When a major item 141 is added, key sentences 142 are added as needed in association with this major item 141 according to the progress of the discussion.

[0230] Also, in Figure 32, the key sentences 142 are not displayed by operating the open / close button 145. In this state, the participant can comprehensively check only the major items 141.

[0231] Figure 33 shows specific examples of major items and key sentences. The major item 151 in Figure 33 is "Proposing a product experience", the first key sentence 152 is "Allowing new employees to actually touch the product makes it easier for them to become familiar with it", and the second key sentence 153 is "Workshop where one talks about one's dream several years later". The generation AI system 4 can generate such major items 151 and key sentences 152, 153, and the third display control unit 22 can display them in the AI analysis area 104 as needed. "As needed" means being displayed at the timing when the generation AI system 4 generates (immediately after the AI analysis is completed excluding delays due to communication, etc.). In addition to such real-time usage, as will be described later with reference to Figure 42, it is also possible to use it so that the participant can display the AI analysis results after the meeting.

[0232] Also, when generating key sentences 152 and 153, the generative AI system 4 classifies the content of key sentences 152 and 153 and determines the categories. In FIG. 33, the category 154 of key sentence 152 is determined to be "Comment", and the category 155 of key sentence 153 is determined to be "Proposal". The third display control unit 22 displays categories 154 and 155 in association with key sentences 152 and 153. By checking categories 154 and 155, participants can understand the content of key sentences 152 and 153 or search for the target key sentences. For example, participants can search for key sentences using the category as a key and display a list of key sentences determined to be in the same category.

[0233] In FIG. 33, categories 154 and 155 are arranged at the upper left of key sentences 152 and 153, but the positions of categories 154 and 155 are just examples. Also, the generative AI system 4 may set two or more categories for one key sentence.

[0234] <<Display of major items and key sentences in the whiteboard work area>> FIG. 34 shows major item 162 and key sentences 163 and 164 displayed in the whiteboard work area 102 when the placement button 146 is pressed. When the participant presses the placement button 146, major item 162 and key sentences 163 and 164 are displayed in the whiteboard work area 102. In FIG. 34, two key sentences 163 and 164 are arranged in parallel below major item 162. Also, major item 162 and the two key sentences 163 and 164 are surrounded by a frame 161. The frame 161 enables the participant to easily grasp that major item 162 and the two key sentences 163 and 164 are in one group (highly related).

[0235] The shape of the frame 161 can be determined by the generative AI system 4. The shape of the frame 161 in Fig. 34 is rectangular. It is known that the shape of the frame, such as a curve, a rectangle, or freehand, can change the impression given to the participants. Therefore, by having the generative AI system 4 determine the shape of the frame 161 based on the content of the major item 162 and the key sentences 163, 164, the participants can receive an impression corresponding to the content. The following are the elements for the generative AI system 4 to determine the shape of the frame 161. · The major item 162 and the key sentences 163, 164 · Only the major item 162 · Both of the key sentences 163, 164 · The key sentences judged to be important among the multiple key sentences 163, 164 · The categories of both of the key sentences 163, 164 · The categories of the key sentences judged to be important among the multiple key sentences 163, 164 Not limited to the shape of the frame 161, the type of frame line (solid line, dotted line, dashed line, double line, etc.), the color of the frame, the thickness of the frame, the color inside the frame, etc. can also be determined by the generative AI system 4 based on at least one of the major item 162 and the key sentences 163, 164.

[0236] Also, regarding the fonts of the major item 162 and the key sentences 163, 164, the generative AI system 4 can determine them based on at least one of the major item 162 and the key sentences 163, 164. Note that the font may be (i) commonly determined for the major item 162 and the key sentences 163, 164 inside the frame, or (ii) determined individually for the major item 162 by the major item 162 and for the key sentences 163, 164 by the key sentences 163, 164 respectively. (i) In the case of common determination, the elements for judgment can be considered the same as those for determining the shape of the frame. (ii) The elements for judgment in individual cases are as follows: the font of major item 162 is determined by major item 162, the font of key sentence 163 is determined by key sentence 163 or category 165, and the font of key sentence 164 is determined by key sentence 164 or category 166. When using the same font for both key sentences 163 and 164, it is determined by both key sentences 163 and 164, the key sentence judged to be important among multiple key sentences 163 and 164, or the category.

[0237] The shape of the frame, the type of the frame line, the color of the frame, the thickness of the frame, the color inside the frame, and the font are referred to as "frame information". The frame information may be determined for each frame displayed in the whiteboard work area 102, or may be determined commonly for all frames. When determined commonly, the generation AI system 4 unifies to, for example, the most common frame information.

[0238] Also, as shown in FIG. 34, an illustration 167 is displayed at the upper right of the frame 161 displayed in the whiteboard work area 102. This illustration 167 is the same as the one displayed in the AI analysis area 104 based on major item 151. The participants can judge the content of the discussion inside the frame by the illustration 167. Also, in FIG. 34, an illustration 168 is displayed at the lower right of key sentence 164. The following can be considered as the display modes of the illustrations based on key sentences 163 and 164. Note that the illustration displayed inside the frame may be the same as that in the AI analysis area 104. · Display illustrations for all key sentences 163 and 164. · Display illustrations only for the key sentence judged to be the most important. · Display illustrations for all key sentences judged to have a certain level of importance or higher. · Do not display any illustrations for the key sentences.

[0239] In addition, in FIG. 34, category 165 is displayed above the left of key sentence 163, and category 166 is displayed above the left of key sentence 164. The following can be considered as the display modes of the illustrations based on key sentences 163 and 164. Note that the category displayed within the frame may be the same as that in the AI analysis area 104. · Display categories for all key sentences 163 and 164. · Display categories only for the key sentence determined to be the most important. · Display categories for key sentences determined to have a certain degree of importance or more. · Do not display any categories for the key sentences.

[0240] Also, in FIG. 34, only the major item 162 and the key sentences 163 and 164 are arranged within the frame 161, but the generation AI system 4 determines the relationship between the major item 162 and the key sentences 163 and 164, and the relationship between the key sentences 163 and 164, and may connect between the major item 162 and the key sentence 163, or between the key sentences 163 and 164. For example, the connection relationship and arrangement are selected according to the analysis contents of the major item 162 and the key sentences 163 and 164 as in the arrangement information table of FIG. 9.

[0241] Note that in FIG. 34, the generation AI system 4 determines the frame information and the like, and the terminal device 1 displays the major item and the key sentence. However, the participant may display the set of the major item and the key sentence in the AI analysis area 104 on the whiteboard work area 102 by drag - and - drop or the like. In this case, without the intervention of the generation AI system 4 or suppressing the intervention, the second display control unit 17 may simply display the major item 162 and the key sentences 163 and 164 surrounded by a rectangular frame. By doing so, the communication delay occurring among the terminal device 1, the information processing device 3, and the generation AI system 4 can be suppressed.

[0242] In addition, the participant may manually input characters, illustrations, tags, etc. directly into the frame 161.

[0243] FIG. 35 shows an example of a whiteboard working area 102 in which a plurality of frames are displayed. In FIG. 35, thirteen frames, frames 283a to 283m, are displayed. As shown in FIG. 35, for each set of a major item and one or more key sentences, the participant can display the major item and the key sentences within the frame. A number 284 is displayed within each of the frames 283a to 283m. This indicates the order of the frames displayed in the whiteboard working area 102. By checking the number 284 and viewing the frames 283a to 283m in order, the user can grasp the flow of the discussion. In FIG. 35, the shapes of the frames 283a to 283m are the same. This is presumably because, for example, the generative AI system 4 has determined the common frame shape, frame line type, frame color, frame thickness, color inside the frame, and font based on all "sets of a major item and one or more key sentences".

[0244] FIG. 36 shows a whiteboard working area 102 in which frames are displayed in different manners according to sets of a major item and one or more key sentences. As shown in FIG. 36, the frame 288 has no frame line. Although the figure is in black and white, the range of the frame 288 is indicated by the coloring inside the frame 288a. The frame 289 is in a freehand shape and is a thick solid line. The frame 290 is in a freehand shape and is a dotted line. The frame 291 is rectangular and is a solid line. The frame 292 is circular and is a solid line.

[0245] Also, the interiors of the frames 288a to 288e are colored with different colors respectively. The interior of the frame 293 is hatched. Also, for example, the font of the frame 288d is different from the font of the frame 289. Also, the background of the key sentences inside the frame can also be colored, and the generative AI system 4 can determine this from the content of the key sentences.

[0246] In this way, the terminal device 1 can display the major item and the key sentences in frames of various manners (shape, frame line type, frame color, frame thickness, color inside the frame, font) in the same whiteboard working area 102. It becomes easier for the participants to visually grasp what the major item and the key sentences are about.

[0247] <Method for Arranging Major Items and Key Sentences Using a Content Arrangement Template> Next, with reference to FIGS. 37 and 38, the content arrangement template will be described. FIG. 37 shows a selection screen 170 of the content arrangement template. The content arrangement template is a tool for efficiently analyzing the content (language information) of a meeting. In other words, it is a mold for arranging major items and key sentences that is prepared for grasping trends and characteristics, efficient transmission, and intuitive understanding of language information. The arbitrary major items and key sentences are all the past major items and key sentences displayed in the AI analysis area 104 from the start to the end of the meeting. It doesn't matter whether the major items and key sentences are displayed in the AI analysis area 104 or in the whiteboard work area 102.

[0248] In FIG. 37, candidates for a plurality of content arrangement templates 171 are displayed. The generation AI system 4 proposes one or more content arrangement templates 171 to the participants in consideration of the language information up to now. The content arrangement template proposed by the generation AI system 4 is surrounded by a red frame 169. There may be a plurality of proposed content arrangement templates. In this way, by emphasizing the content arrangement template 171 proposed by the generation AI system 4, the participants can distinguish the content arrangement template 171 proposed by the generation AI system 4. The participants can use the content arrangement template surrounded by the red frame 169 proposed by the generation AI system 4, or can select an arbitrary content arrangement template from the selection screen 170 of the content arrangement template. Also, instead of the participants making the final selection, the content arrangement template determined by the generation AI system 4 to be one may be displayed in the whiteboard work area 102.

[0249] The selection screen 170 of the content layout template is automatically displayed when the generation AI system 4 determines that there is an appropriate content layout template. Alternatively, it may be displayed when the participant inputs an operation to request a content layout template. For example, a template button 147 is displayed in the upper right of the whiteboard work area 102. When the participant presses the template button 147, the generation AI system 4 proposes an appropriate content layout template.

[0250] In addition, in FIG. 37, a frame 161 including the major item 162 and the key sentences 163, 164 is already displayed in the whiteboard work area 102. However, before the display of the content layout template, it is not necessary for the major item 162 and the key sentences 163, 164 to be displayed in the whiteboard work area 102. The content layout template can be displayed by the terminal device 1 when there is nothing in the whiteboard work area 102.

[0251] FIG. 38 shows an example of the content layout template 172 displayed in the whiteboard work area 102. The content layout template 172 is provided with placement columns 173 to 175 for dragging and dropping any major item or key sentence. The participant can drag and drop any major item and key sentence into any of the placement columns 173 to 175. In FIG. 38, the major item 162 is dragged and dropped into the placement column 175, the key sentence 164 is dragged and dropped into the placement column 174, and the key sentence 152 is dragged and dropped into the placement column 173.

[0252] In this way, since a content layout template corresponding to the content of the discussion is proposed, the participant can effectively summarize the content of the discussion using the major item and the key sentences. Note that the participant can directly input characters, etc. into any placement column instead of dragging and dropping.

[0253] <Display of AI analysis results by the generation AI system> Next, with reference to FIG. 39, the process in which the terminal device 1 displays the AI analysis result generated by the generative AI system 4 through AI analysis of language information will be described. FIG. 39 is a sequence diagram for explaining the process in which the terminal device 1 displays the AI analysis result generated by analyzing the language information by the generative AI system 4 in the AI analysis area 104. Note that, by the process of FIG. 16, a certain amount of language information has already been accumulated in the conversation accumulation area 101.

[0254] S201: The participant operates the terminal device 1a to turn on the AI analysis function (toggle button 131). Also, the participant selects tab 133. The operation reception unit 18 of the terminal device 1a receives this operation.

[0255] S202: The first communication unit 11 of the terminal device 1a transmits the start of AI analysis to the information processing device 3 due to the turning on of the toggle button 131.

[0256] S203: The second communication unit 31 of the information processing device 3 receives the start of AI analysis, and the third communication unit 38 transmits the start of AI analysis to the generative AI system 4. As a result, the generative AI system 4 starts AI analysis of the language information already accumulated in the conversation accumulation area 101 and subsequent language information.

[0257] After that, each participant participating in the meeting makes a speech, and steps S101 to S112 in FIG. 16 are repeatedly executed.

[0258] S204: The analysis information acquisition unit 37 of the information processing device 3 transmits language information (including the speaker and date / time of each language information) to the generative AI system 4. It is assumed that the analysis information acquisition unit 37 transmits an amount of language information suitable for AI analysis that enables context analysis to the generative AI system 4. For example, the analysis information acquisition unit 37 may transmit language information to the generative AI system 4 each time a certain number of language information is accumulated.

[0259] Further, the analysis information acquisition unit 37 may transmit all the language information from when the last major item is determined until the next major item is determined to the generation AI system 4. That is, the language information from when the last major item is determined until immediately before the present is transmitted repeatedly, and in addition to this, new language information is transmitted. Thereby, the generation AI system 4 can detect that the topic has changed and determine a new major item.

[0260] Also, the language information may be transmitted to the generation AI system 4 sequentially without being stored. In this case, it is assumed that the generation AI system 4 has a function of storing the language information, and the generation AI system 4 determines the change in the topic.

[0261] S205: The third communication unit 38 of the information processing apparatus 3 transmits the language information, the speaker, and the date and time to the generation AI system 4 together with the AI analysis request.

[0262] S206: The generation AI system 4 performs AI analysis on the language information and the like, generates an AI analysis result, and returns it to the information processing apparatus 3. The AI analysis result is, for example, a major item and a key sentence, a category of the key sentence, an illustration of the major item, an illustration of the key sentence, and the like. The analysis information acquisition unit 37 of the information processing apparatus 3 acquires the AI analysis result via the third communication unit 38.

[0263] S207, S208: The second communication unit 31 of the information processing apparatus 3 transmits the AI analysis result to the terminal devices 1a to 1c.

[0264] S209, S210: The first communication unit 11 of the terminal devices 1a to 1c receives the AI analysis result, and the third display control unit 22 displays the AI analysis result in the AI analysis area 104. That is, the major item and the key sentence are displayed in the AI analysis area 104, an illustration is displayed in association with the major item, and a category and an illustration are displayed in association with the key sentence.

[0265] As described above, as shown in FIG. 33, the major item 151, the key sentences 152, 153, the illustrations 157, 158, and the categories 154, 155 are displayed in the AI analysis area 104.

[0266] Next, referring to FIG. 40, the flow of the process of displaying the major items and key sentences in the whiteboard work area 102 will be described. FIG. 40 is a sequence diagram for explaining the flow of the process in which the terminal device 1 displays the major items and key sentences in the whiteboard work area 102.

[0267] S231: The participant operates the terminal device 1a to press the placement button 146. The operation reception unit 18 of the terminal device 1a receives the pressing of the placement button 146.

[0268] S232: The first communication unit 11 of the terminal device 1a transmits a request for frame information and position information to the information processing device 3 due to the pressing of the placement button 146. The position information is information for arranging the major items and key sentences in the whiteboard work area 102. Therefore, the information transmitted to the information processing device 3 includes the major items and key sentences. Alternatively, instead of the major items and key sentences themselves, information (such as an ID) for specifying the major items and key sentences may be included.

[0269] S233: The second communication unit 31 of the information processing apparatus 3 receives a request for frame information and position information, and the third communication unit 38 transmits the request for frame information and position information to the generation AI system 4. The generation AI system 4 determines frame information (frame shape, frame line type, frame color, frame thickness, color inside the frame, font) based on at least one of the major item and the key sentence. Further, the generation AI system 4 determines the size of the frame based on the size of the major item (size of the circumscribed rectangle), the size of the key sentence (size of the circumscribed rectangle), and the number. The generation AI system 4 determines the size of the frame to be the size that includes the major item and all the key sentences. Also, the generation AI system 4 determines the position information of the major item and determines the position information of the key sentences within the frame based on the number of the key sentences. As a simple method, the generation AI system 4 determines the position information to arrange the major item at the topmost position and arrange the key sentences in two columns in time series below it. Note that the position information may be a relative position based on a frame with, for example, the upper left corner of the frame as the origin, or an absolute position indicating the position in the whiteboard work area 102. In the case of the relative position, the participant moves the frame to the empty space in the whiteboard work area 102. In the case of the absolute position, since the generation AI system 4 determines the empty space in the whiteboard work area 102, the second display control unit 17 arranges the frame based on the absolute position, and thus the frame is displayed in the empty space.

[0270] In addition, the generative AI system 4 determines the illustration 167 placed in the upper right of the frame. However, this illustration 167 has already been determined based on the content of the major item (corresponding to illustration 156). Therefore, it is not necessary for the generative AI system 4 to newly determine it. In this case, the second display control unit 17 of the terminal device 1 displays the illustration 156 in the conversation storage area 101 in the upper right of the frame 161 in the whiteboard work area 102. Also, the generative AI system 4 determines the illustration 168 to be displayed in association with the key sentences 163 and 164. This illustration 168 has already been determined based on the content of the key sentence 164. Therefore, it is not necessary for the generative AI system 4 to newly determine it. In this case, the second display control unit 17 of the terminal device 1 displays the illustration 158 in the conversation storage area 101 around the key sentence 164 as the illustration 168.

[0271] S234: The generative AI system 4 returns the frame information, the position information of the major item and the key sentences, the category, and the illustration to the information processing device 3. The analysis information acquisition unit 37 of the information processing device 3 acquires these via the third communication unit 38.

[0272] S235, S236: The second communication unit 31 of the information processing device 3 transmits the frame information, the position information of the major item and the key sentences, the category, and the illustration to the terminal devices 1a to 1c.

[0273] S237, S238: The first communication units 11 of the terminal devices 1a to 1c receive the frame information, the position information of the major item and the key sentences, and the illustration. The second display control unit 17 forms the frame 161 in the whiteboard work area 102 and displays the major item 162 and the key sentences 163 and 164 inside the frame based on the position information. Also, the second display control unit 17 displays the illustration 167 determined based on the major item in the upper right of the frame 161, displays the category 165 for the key sentence 163, and displays the category 166 and the illustration 168 for the key sentence 164.

[0274] As described above, as shown in FIG. 34, the major item 162 and the key sentences 163 and 164 are displayed inside the frame 161 in the whiteboard work area 102.

[0275] In the process shown in FIG. 40, the terminal devices 1a to 1c display the same major items and key sentences within the frame. However, it is also possible that only the terminal device 1a that has pressed the arrangement button 146 displays the major items and key sentences within the frame. That is, each participant can select the major items and key sentences to be displayed in the whiteboard work area 102 using the arrangement button 146.

[0276] Next, referring to FIG. 41, the process flow for displaying the content arrangement template will be described. FIG. 41 is a sequence diagram for explaining the process flow in which the terminal device 1 displays the content arrangement template.

[0277] S251: The participant inputs an operation to display the content arrangement template on the terminal device 1a (pressing the template button 147). The operation reception unit 18 of the terminal device 1a receives this operation. The content arrangement template may be automatically proposed by the generation AI system 4 based on the determination of past language information.

[0278] S252: The first communication unit 11 of the terminal device 1a transmits a content arrangement template request to the information processing device 3 when the template button 147 is pressed.

[0279] S253: The second communication unit 31 of the information processing device 3 receives the content arrangement template request, and the third communication unit 38 transmits the content arrangement template request to the generation AI system 4. Since language information is required for the proposal of the content arrangement template, the information processing device 3 transmits all or at least part of the language information stored in the conversation storage area 101 to the generation AI system 4.

[0280] S254: The generation AI system 4 determines one or more content placement templates to propose based on language information. It is advisable that the information processing device 3 pre-transmits all the content placement templates to the generation AI system 4 and requests it to select the content placement templates to propose from them. The generation AI system 4 returns the proposal of the content placement template to the information processing device 3. The proposal of the content placement template includes all the selectable content placement templates and the identification information of the one or more content placement templates to propose. The information processing device 3 holds all the selectable content placement templates and may transmit them to the terminal device 1 together with the one or more content placement templates proposed by the generation AI system 4.

[0281] S255, S256: The analysis information acquisition unit 37 of the information processing device 3 acquires the proposal of the content placement template via the third communication unit 38. The second communication unit 31 of the information processing device 3 transmits the screen information of the content placement template selection screen 170 to the terminal devices 1a to 1c. On the content placement template selection screen 170, the content placement templates proposed by the generation AI system 4 are highlighted with a red frame or the like.

[0282] S257, S258: The first communication unit 11 of the terminal devices 1a to 1c receives the screen information of the content placement template selection screen 170, and the second display control unit 17 causes the content placement template selection screen 170 to be displayed on the whiteboard work area 102.

[0283] In the process of FIG. 41, although the terminal devices 1a to 1c each display the content placement template selection screen 170, it is also possible that only the terminal device 1a (for example, the terminal device 1 of the facilitator) that requested the content placement template selection screen 170 displays the content placement template selection screen 170. As described above, as shown in FIG. 37, the content placement template selection screen 170 is displayed.

[0284] S259: The participant selects a content placement template from the content placement template selection screen 170. Alternatively, if only one content placement template proposed by the generation AI system 4 is determined, the received content placement template is displayed even if the participant does not select it.

[0285] As described with reference to FIG. 38, the participant can drag and drop any major item or key sentence onto the content placement template to summarize the content of the discussion. Since the content placement template is proposed according to the content of the discussion, it is expected that the participant can easily summarize the content of the discussion.

[0286] <AI Analysis after Meeting End> In this embodiment, an embodiment in which AI analysis results are displayed in real time during a meeting has been described. However, there may be a case where the AI analysis results are to be displayed after the meeting ends. For example, during the meeting, each participant works in their respective whiteboard work area 102, and after the meeting ends, it becomes possible to share the AI analysis results and review the content of the meeting.

[0287] When performing AI analysis in retrospect, at least one of the terminal device 1 and the information processing device 3 stores language information. After the participant operates the toggle button 131 to turn on the AI analysis function, the information processing device 3 transmits the language information to the generation AI system 4.

[0288] FIG. 42 is a sequence diagram for explaining the process in which the terminal device 1 displays the AI analysis results generated by the generation AI system 4 analyzing the language information after the meeting ends. Note that in the description of FIG. 42, the differences from FIG. 39 may mainly be explained.

[0289] First, each participant participating in the meeting makes a speech, and steps S101 to S112 in FIG. 16 are repeatedly executed. Thereafter, it is assumed that the meeting has ended. However, it is also possible to turn on the AI analysis function during the meeting.

[0290] S271: The participant operates the terminal device 1a to turn on the AI analysis function (toggle button 131). Also, the participant selects tab 133. The operation reception unit 18 of the terminal device 1a receives this operation.

[0291] S272: The first communication unit 11 of the terminal device 1a transmits the start of AI analysis to the information processing device 3 due to the toggling on of the toggle button 131.

[0292] S273: The second communication unit 31 of the information processing device 3 receives the start of AI analysis, and the third communication unit 38 transmits the start of AI analysis to the generated AI system 4.

[0293] S274: Subsequently, the second communication unit 31 of the information processing device 3 transmits the language information of the entire meeting accumulated or received from the terminal device 1 to the generated AI system 4. The second communication unit 31 may transmit all the language information to the generated AI system 4 at once, or may transmit the language information little by little as in the middle of the meeting (see S204).

[0294] S275 - S279: The subsequent processing may be the same as that in FIG. 39. As described above, as shown in FIG. 33, the major item 151 and the key sentences 152, 153 are displayed in the AI analysis area 104. After this, the participant can press the placement button 146 to display the major item 162 and the key sentences 163, 164 in the whiteboard work area 102, or can display the content placement template 172.

[0295] In this way, the participant can turn on the AI analysis function after (or during) the end of the meeting and review the content of the meeting while viewing the AI analysis results. The operation to display the AI analysis results can be accepted before the start of communication, during communication, or after the end of communication.

[0296] <Main effects> In addition to the effects of the first embodiment, the communication system 5 of this embodiment can reduce the operation burden on the participant for operating the setting screen.

[0297] <Other application examples> As described above, the best mode for carrying out the present invention has been described using examples. However, the present invention is not limited to such examples, and various modifications and substitutions can be made without departing from the gist of the present invention.

[0298] For example, in the present embodiment, mainly voice data is converted into language information. However, the language information may be chat (text data) input by the participant using a keyboard.

[0299] In the present embodiment, the terminal device 1 transmits language information to the information processing device 3, and the information processing device 3 transmits the language information to the generation AI system 4, so that the analysis information acquired by the information processing device 3 is transmitted to the terminal device 1. However, the terminal device 1 may transmit the language information within the specified range to the generation AI system 4 and directly acquire the analysis information from the generation AI system 4. The terminal device 1 executes a web application or a native application, performs necessary processing on the terminal device 1 side, and can display the analysis information on the whiteboard work area 102. For example, when the terminal device 1 is used at one location (when the terminal device 1 is an electronic blackboard), such a usage may be adopted.

[0300] For example, in the present embodiment, the information display screen 100 is displayed on the display 506. However, the projector may display the information display screen 100. In this case, the communication system 5 detects the coordinates of the tip of the pen for handwriting by ultrasonic waves. The pen emits ultrasonic waves together with light emission, and the terminal device 1 calculates the distance based on the arrival time of the ultrasonic waves. The terminal device 1 can specify the position of the pen based on the direction and distance, and the projector draws (projects) the trajectory of the pen as stroke data.

[0301] Also, there may be only one terminal device 1. In this case, the participant can use the terminal device 1 as a tool for summarizing or recording their thoughts.

[0302] In addition, the configuration examples such as FIG. 4 are divided according to the main functions in order to facilitate the understanding of the processing by the terminal device 1 and the information processing device 3. The present invention is not limited by the way of dividing the processing units and the names thereof. The processing of the terminal device 1 and the information processing device 3 can be further divided into more processing units according to the processing content. Also, it can be divided such that one processing unit includes more processing.

[0303] In addition, the device group described in the embodiments merely shows one of a plurality of computing environments for implementing the embodiments disclosed in this specification. In one embodiment, the information processing device 3 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via an arbitrary type of communication link including a network, a shared memory, etc., and implement the processing disclosed in this specification.

[0304] Furthermore, the information processing device 3 can be configured to share the processing steps disclosed in this embodiment, for example, FIGS. 16, 17, etc. in various combinations. For example, the process executed by a predetermined unit can be executed by a plurality of information processing devices included in the information processing device 3. Also, the information processing device 3 may be integrated into one server device or divided into a plurality of devices.

[0305] Each function of the embodiments described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array) designed to execute each function described above, and devices such as conventional circuit modules.

[0306] In addition, the table in the specification may be generated by the learning effect of machine learning. Here, machine learning is a technology for enabling a computer to acquire learning capabilities similar to those of humans. That is, the computer autonomously generates an algorithm necessary for judgments such as data identification from learning data that is pre-imported, and applies this to new data to make predictions. The learning method for machine learning may be any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. Furthermore, a learning method combining these learning methods may also be used, and the learning method for machine learning is not limited.

[0307] <Supplementary Note> [Supplementary Note 1] A communication system in which an information processing apparatus and one or more terminal devices communicate via a network, wherein the terminal device, has a first display control unit that displays language information based on information input to the terminal device in a time series in a first area of an information display screen of the terminal device; wherein the information processing apparatus, has an analysis information acquisition unit that acquires analysis information obtained by analyzing language information within a specified range among the language information based on information input to the terminal device; wherein the terminal device, has a second display control unit that displays the analysis information acquired by the information processing apparatus in a second area of the information display screen; A communication system having the above. [Supplementary Note 2] wherein the information processing apparatus, has a conversion unit that converts voice information based on voice data input to the terminal device into the language information; The communication system according to Supplementary Note 1, wherein the first display control unit displays the language information converted by the conversion unit in the first area. [Supplementary Note 3] The communication system according to Supplementary Note 1 or 2, wherein the analysis information acquisition unit acquires the analysis information obtained by analyzing the language information with preset analysis content. [Appendix 4] The preset analysis content is the communication system described in Appendix 3, which is to analyze the language information to obtain one or more of the summary, topic, or issue of the language information within the specified range. [Appendix 5] The preset analysis content is the communication system described in Appendix 3, which is to analyze the language information to obtain the information obtained by diverging or converging the language information within the specified range. [Appendix 6] The preset analysis content is the communication system described in Appendix 3, which is the information specifying the analysis content of the language information input in natural language by the participants of the terminal device. [Appendix 7] The information processing device has a communication unit that transmits screen information for receiving the selection of the analysis content to the terminal device, and receives the setting of the analysis content from the terminal device, which is the communication system according to any one of Appendices 3 to 6. [Appendix 8] The language information within the specified range is specified by the start time and the end time with respect to the time associated with the language information, which is the communication system according to any one of Appendices 1 to 6. [Appendix 9] The language information within the specified range is the language information received by the information processing device from the terminal device during a certain time interval, which is the communication system according to any one of Appendices 1 to 6. [Appendix 10] The language information within the specified range is the number of characters of the language information received by the information processing device from the terminal device, which is the communication system according to any one of Appendices 1 to 6. [Appendix 11] The language information within the specified range is the language information from when the speaker of the language information last switched until the next speaker switches, which is the communication system according to any one of Appendices 1 to 6. [Appendix 12] The communication system according to any one of Appendices 1 to 6, wherein the language information within the specified range is the language information from the last time the topic of the language information is switched until the next topic switch. [Appendix 13] The information processing apparatus has a communication unit that transmits screen information for receiving a selection of the range of the language information to the terminal device, The communication system according to any one of Appendices 8 to 12, which receives a setting of the range of the language information from the terminal device. [Appendix 14] The communication system according to any one of Appendices 1 to 13, wherein the second display control unit displays the analysis information in the second area in a display mode corresponding to the type of the analysis information. [Appendix 15] The communication system according to any one of Appendices 1 to 14, wherein the second display control unit emphasizes and displays a specific character string included in the analysis information. [Appendix 16] When the analysis information acquisition unit acquires a plurality of the analysis information, it has an analysis information arrangement unit that determines the arrangement of the plurality of the analysis information based on arrangement information associated with the type of the analysis information, The communication system according to any one of Appendices 1 to 15, wherein the second display control unit displays the analysis information in the second area based on the arrangement determined by the analysis information arrangement unit. [Appendix 17] The communication system according to Appendix 16, wherein the arrangement determined by the analysis information arrangement unit includes a connection line between one analysis information and the other analysis information. [Appendix 18] The communication system according to Appendix 16 or 17, wherein the second display control unit displays the analysis information in an area preset by a participant among the second areas. [Appendix 19] The terminal device has a search unit that acquires an illustration based on the analysis information displayed in the second area, The second display control unit displays the illustration around the analysis information displayed in the second area. The communication system according to any one of Appendices 1 to 18. [Appendix 20] The second display control unit displays the analysis information in a third area different from the first area and the second area. When an operation to move the analysis information selected in the third area to the second area is received, the second display control unit displays the analysis information in the second area. The communication system according to any one of Appendices 1 to 17. [Appendix 21] The terminal device has a search unit that acquires an illustration based on the analysis information displayed in the third area. The second display control unit displays the illustration around the analysis information in the third area. When an operation to move the illustration selected in the third area to the second area is received, the second display control unit displays the illustration in the second area. The communication system according to Appendix 20. [Appendix 22] The analysis information acquisition unit acquires, as the analysis information, information generated based on the analysis result obtained by analyzing the language information based on the information input to the terminal device by a generation AI system. The information processing device has a communication unit that transmits the information generated based on the analysis result to the terminal device. The terminal device has a third display control unit that causes a fourth area in which the information generated based on the analysis result is displayed to be displayed. The third display control unit switches and displays from the first area to the fourth area according to an operation. The communication system according to claim 1. [Appendix 23] The information generated based on the analysis result has first information that is the title of the language information and one or more second information extracted from the language information. The communication system according to appended note 22, wherein one or more sets in which the one or more pieces of second information are associated with the first information are displayed in the fourth region. [Appended note 24] The information generated based on the analysis result includes an illustration related to the second information. The communication system according to appended note 23, wherein the third display control unit displays the illustration associated with the second information one-to-one in the fourth region. [Appended note 25] When an operation to display the entire set in which the first information and the one or more pieces of second information are associated, which is displayed in the fourth region, in the second region is received, The communication system according to appended note 23, wherein the second display control unit displays the first information and the one or more pieces of second information included in the set surrounded by a frame in the second region. [Appended note 26] The terminal device transmits to the information processing device that an operation to display the entire set in the second region has been received. The analysis information acquisition unit designates the first information and the one or more pieces of second information included in the set, and acquires frame information including the shape of the frame from the generation AI system. The communication unit transmits the frame information to the terminal device. The communication system according to appended note 25, wherein the second display control unit displays the first information and the one or more pieces of second information surrounded by a frame having the shape of the frame in the second region. [Appended note 27] The frame information includes one or more of the type of frame line, the color of the frame, the thickness of the frame, the color inside the frame, or the font. The communication system according to appended note 26, wherein the second display control unit performs one or more of displaying the frame with the type of frame line, displaying the frame with the color of the frame, displaying the frame with the thickness of the frame, displaying the inside of the frame with the color inside the frame, or displaying the first information and the one or more pieces of second information with the font. [Appended note 28] The terminal device transmits to the information processing device that it has received an operation to display an arbitrary template for arranging the first information and the one or more second information. The analysis information acquisition unit acquires one or more candidates of the template from the generation AI system by specifying the language information. The communication unit transmits one or more candidates of the template to the terminal device. The second display control unit causes one or more candidates of the template to be displayed in the second area. The communication system according to supplementary note 25, which causes the selected template to be displayed in the second area. [Supplementary note 29] The analysis information acquisition unit acquires the identification information of the most recommended template from the generation AI system. The communication unit transmits the identification information of the most recommended template to the terminal device. The communication system according to supplementary note 28, wherein the second display control unit emphasizes and displays the most recommended template when displaying one or more candidates of the template. [Supplementary note 30] The terminal device transmits to the information processing device that it has received an operation to display the information generated based on the analysis result. The analysis information acquisition unit acquires the information generated based on the analysis result from the generation AI system according to the operation. The operation to display the information generated based on the analysis result can be received before the start of communication, during communication, or after the end of communication. The communication system according to any one of supplementary notes 22 to 29. [Supplementary note 31] When the terminal device receives an operation to display the information generated based on the analysis result after the end of communication, The terminal device transmits to the information processing device that it has received an operation to display the information generated based on the analysis result. The analysis information acquisition unit acquires, in response to the operation, information generated based on the analysis result for the language information from the start to the end of the communication from the generation AI system, The communication unit transmits the information generated based on the analysis result to the terminal device, The terminal device The third display control unit causes the information generated based on the analysis result to be displayed in the fourth area, The communication system according to Supplementary Note 30.

Explanation of Signs

[0308] 1 Terminal device 3 Information processing device 4 Generation AI system 5 Communication system

Prior Art Documents

Patent Documents

[0309]

Patent Document 1

Claims

1. A communication system in which an information processing device and one or more terminal devices communicate with each other via a network, The terminal device a first display control unit that displays language information based on information input to the terminal device in a time series in a first area of ​​an information display screen of the terminal device; The information processing device includes: an analysis information acquisition unit that acquires analysis information obtained by analyzing a specified range of the language information based on information input to the terminal device; The terminal device a second display control unit that displays the analysis information acquired by the information processing device in a second area of ​​the information display screen; A communication system having the above configuration.

2. The information processing device includes: a conversion unit that converts voice information based on voice data input to the terminal device into the language information; The communication system according to claim 1 , wherein the first display control unit displays the language information converted by the conversion unit in the first area.

3. The communication system according to claim 1 or 2, wherein the analysis information acquisition unit acquires the analysis information obtained by analyzing the language information with a preset analysis content.

4. The communication system according to claim 3 , wherein the predetermined analysis content is to analyze the linguistic information to obtain one or more of a summary, a topic, or an agenda of the linguistic information in a specified range.

5. The communication system according to claim 3 , wherein the predetermined analysis content is to analyze the linguistic information and obtain divergent or convergent information of the linguistic information within a specified range.

6. 4. The communication system according to claim 3, wherein the predetermined analysis content is information that specifies analysis content of the linguistic information input in a natural language by a participant of the terminal device.

7. The information processing device includes: a communication unit that transmits screen information for accepting a selection of the analysis content to the terminal device; The communication system according to claim 3 , wherein the analysis content is set from the terminal device.

8. 2. The communication system according to claim 1, wherein the specified range of language information is specified by a start time and an end time with respect to a time associated with the language information.

9. The communication system according to claim 1 , wherein the language information in the specified range is the language information received by the information processing device from the terminal device during a certain time interval.

10. The communication system according to claim 1 , wherein the specified range of language information is a number of characters of the language information received by the information processing device from the terminal device.

11. 2. The communication system according to claim 1, wherein the specified range of language information is the language information from the last speaker change of the language information to the next speaker change.

12. 2. The communication system according to claim 1, wherein the specified range of linguistic information is the linguistic information from the last change in topic of the linguistic information to the next change in topic.

13. The information processing device includes: a communication unit that transmits screen information for receiving a selection of the range of the language information to the terminal device; The communication system according to claim 8 , further comprising: a setting of the range of the language information received from the terminal device.

14. The communication system according to claim 1 , wherein the second display control unit displays the analysis information in the second area in a display mode according to a type of the analysis information.

15. The communication system according to claim 1 , wherein the second display control unit displays a specific character string included in the analysis information in an emphasized manner.

16. an analysis information arrangement unit that, when the analysis information acquisition unit acquires a plurality of pieces of analysis information, determines an arrangement of the plurality of pieces of analysis information based on arrangement information associated with a type of the analysis information; The communication system according to claim 1 , wherein the second display control unit displays the analysis information in the second area based on the layout determined by the analysis information layout unit.

17. 17. The communication system according to claim 16, wherein the arrangement determined by the analyzed information arrangement unit includes a connection between one analyzed information piece and another analyzed information piece.

18. The communication system according to claim 16 or 17, wherein the second display control unit displays the analysis information in an area of ​​the second area that is preset by a participant.

19. The terminal device a search unit for acquiring an illustration based on the analysis information displayed in the second area; The communication system according to claim 1 , wherein the second display control unit displays the illustration around the analysis information displayed in the second area.

20. the second display control unit displays the analysis information in a third area different from the first area and the second area; The communication system of claim 1, wherein when an operation to move the analysis information selected in the third area to the second area is received, the second display control unit displays the analysis information in the second area.

21. The terminal device a search unit for acquiring an illustration based on the analysis information displayed in the third area; the second display control unit displays the illustration around the analysis information in the third area; The communication system described in claim 20, wherein when an operation to move the illustration selected in the third area to the second area is received, the second display control unit displays the illustration in the second area.

22. The analysis information acquisition unit acquires, as the analysis information, information generated based on an analysis result obtained by analyzing the language information based on the information input to the terminal device by a generation AI system, the information processing device has a communication unit that transmits information generated based on the analysis result to the terminal device; the terminal device has a third display control unit that displays a fourth area in which information generated based on the analysis result is displayed; The third display control unit switches the display from the first area to the fourth area in response to an operation. The communication system according to claim 1 .

23. The information generated based on the analysis result includes first information that is a topic of the linguistic information and one or more second information extracted from the linguistic information; The communication system according to claim 22 , wherein the fourth area displays one or more pairs of the first information associated with the one or more second information.

24. The information generated based on the analysis result includes an illustration related to the second information, The communication system according to claim 23 , wherein the third display control unit causes the illustration in one-to-one correspondence with the second information to be displayed in the fourth area.

25. When an operation for displaying, in the second area, the entire set of the first information and the one or more pieces of second information associated with each other, which is displayed in the fourth area, is received, The communication system according to claim 23 , wherein the second display control unit causes the first information and the one or more pieces of second information included in the set to be displayed in a frame in the second area.

26. the terminal device transmits to the information processing device a notification that an operation to display the entire set in the second area has been accepted; The analysis information acquisition unit acquires frame information including a shape of a frame from the generation AI system by specifying the first information and the one or more second information included in the set, The communication unit transmits the frame information to the terminal device, The communication system according to claim 25 , wherein the second display control unit causes the first information and the one or more pieces of second information surrounded by a frame having the shape of the frame to be displayed in the second area.

27. The frame information includes one or more of a frame type, a frame color, a frame thickness, a frame color, or a font, The communication system of claim 26, wherein the second display control unit performs one or more of the following: displaying the frame in the frame line type, displaying the frame in the frame color, displaying the frame in the frame thickness, displaying the inside of the frame in the frame color, or displaying the first information and the one or more second information in the font.

28. the terminal device transmits to the information processing device a notification that an operation for displaying an arbitrary template for arranging the first information and the one or more pieces of second information has been accepted; The analysis information acquisition unit specifies the language information and acquires one or more candidates for the template from the generation AI system; The communication unit transmits one or more candidates for the template to the terminal device; the second display control unit displays one or more candidates for the template in the second area; 26. The communication system according to claim 25, wherein the template whose selection has been accepted is displayed in the second area.

29. The analysis information acquisition unit acquires identification information of the most recommended template from the generation AI system, The communication unit transmits identification information of the most recommended template to the terminal device; The communication system according to claim 28 , wherein the second display control unit, when displaying the one or more template candidates, highlights the most recommended template.

30. the terminal device transmits to the information processing device a notification that an operation to display information generated based on the analysis result has been accepted; The analysis information acquisition unit acquires information generated based on the analysis result from the generation AI system in response to the operation, A communication system according to any one of claims 22 to 29, wherein an operation to display information generated based on the analysis results can be accepted before the start of communication, during communication, or after the end of communication.

31. When the terminal device receives an operation to display information generated based on the analysis result after the communication is ended, the terminal device transmits to the information processing device a notification that an operation to display information generated based on the analysis result has been accepted; The analysis information acquisition unit acquires information generated based on the analysis result of the language information from the start to the end of the communication from the generation AI system in response to the operation, The communication unit transmits information generated based on the analysis result to the terminal device, The terminal device The third display control unit displays information generated based on the analysis result in the fourth area.

31. A communication system according to claim 30.

32. An information processing device that communicates with one or more terminal devices via a network, a communication unit that receives information regarding language information based on information input to the terminal device; an analysis information acquisition unit that acquires analysis information obtained by analyzing the language information; The communication unit is an information processing device that transmits to the terminal device the language information displayed in chronological order in a first area of ​​a screen displayed by the terminal device, and the analysis information displayed in a second area of ​​the screen.

33. A terminal device that communicates with an information processing device via a network, a first display control unit that displays language information based on information input to the terminal device in a time series in a first area of ​​an information display screen of the terminal device; a communication unit that receives analysis information obtained by analyzing language information within a specified range from the information processing device; a second display control unit that displays the analysis information in a second area of ​​the information display screen; A terminal device having the above configuration.

34. An information processing method performed by a communication system in which an information processing device and one or more terminal devices communicate via a network, comprising: The terminal device performing a process of displaying language information based on information input to the terminal device in a time series in a first area of ​​an information display screen of the terminal device; The information processing device includes: performing a process of acquiring analyzed information obtained by analyzing a specified range of language information from the language information based on information input to the terminal device; The terminal device The information processing method includes processing for displaying the acquired analysis information in a second area of ​​the information display screen.

35. An information processing device that communicates with one or more terminal devices via a network, a communication unit that receives information regarding language information based on information input to the terminal device; The linguistic information is analyzed, and the linguistic information is acquired as an analysis information acquisition unit. The communication unit is a program that transmits to the terminal device the language information displayed in chronological order in a first area of ​​a screen displayed by the terminal device, and the analysis information displayed in a second area of ​​the screen.

36. A terminal device that communicates with the information processing device via a network, a first display control unit that displays language information based on information input to the terminal device in a time series in a first area of ​​an information display screen of the terminal device; a communication unit that receives analysis information obtained by analyzing language information within a specified range from the information processing device; a second display control unit that displays the analysis information in a second area of ​​the information display screen; A program to function as a

Images