Information processing device and information processing method

Abstract

An information processing device according to this disclosure includes: an estimation unit for estimating a topic of a conversation in a group from statement of a participant belonging to the group, the group being obtained by dividing a plurality of participants participating in a remote meeting into a plurality of groups in which the participants can have a conversation; and a notification unit for notifying, if a topic of a second group other than a first group to which one participant belongs to and preference of the one participant meet a condition relating to similarity, the one participant of the second group.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] In recent years, remote meetings (also referred to as "remote conferences") among multiple users have been held by communicating voice and images via the Internet, such as Web conferences. Further, a technique for dividing users participating in such a remote conference (also referred to as "participants") into a plurality of groups is known (for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, it is not always possible to notify appropriate information to participants participating in a remote conference. In the prior art, after division into groups, it is difficult for each user to recognize the situation outside the group to which he / she belongs. Therefore, it is desired to notify appropriate information to participants participating in a remote conference.

[0005] Therefore, this disclosure proposes an information processing device and an information processing method that can notify participants of a remote meeting of appropriate information. [Means for solving the problem]

[0006] To solve the above problems, one form of information processing device according to the present disclosure comprises: an estimation unit that allows multiple participants participating in a remote meeting to be divided into multiple groups, enabling conversations within each group, and estimates the topic of conversation in each group from the statements of the participants belonging to the group; and a notification unit that notifies the first participant of the second group if the topic of conversation in a second group (a group other than the first group to which the first participant belongs) satisfies the conditions regarding similarity between the first participant's preferences and the second group. [Brief explanation of the drawing]

[0007] [Figure 1] This figure shows an example of information processing according to the embodiments of this disclosure. [Figure 2] This figure shows an example of a display for bystanders. [Figure 3] This figure shows another example of a display for bystanders. [Figure 4] This figure shows an example of how information is displayed to the parties involved. [Figure 5] This figure shows an example configuration of a remote meeting system according to the embodiment of this disclosure. [Figure 6] This figure shows an example configuration of a remote meeting server according to an embodiment of this disclosure. [Figure 7] This figure shows an example of a group information storage unit according to an embodiment of this disclosure. [Figure 8] This figure shows an example configuration of a terminal device according to the embodiments of this disclosure. [Figure 9] This is a flowchart showing the processing procedure of an information processing device according to an embodiment of this disclosure. [Figure 10] This is a flowchart showing the processing procedure in a terminal device according to an embodiment of this disclosure. [Figure 11]It is a flowchart showing the processing procedure of a remote meeting server according to an embodiment of the present disclosure. [Figure 12] It is a flowchart showing a matching processing procedure. [Figure 13] It is a diagram showing an example of a display mode. [Figure 14] It is a hardware configuration diagram showing an example of a computer that realizes the functions of an information processing apparatus.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. Note that the information processing apparatus and the information processing method according to the present application are not limited by this embodiment. Also, in each of the following embodiments, the same parts are denoted by the same reference numerals, and redundant explanations are omitted.

[0009] The present disclosure will be described according to the following item order. 1. Embodiment 1-1. Outline of Information Processing According to Embodiment of the Present Disclosure 1-1-1. Display Example with Recommendation Group Notification 1-1-2. Display Example without Recommendation Group Notification 1-1-3. Background and Effects, etc. 1-1-4. User Participation Status 1-1-5. Groups to be Presented 1-1-6. Notification Mode 1-1-7. Group Movement, etc. 1-2. Configuration of Remote Meeting System According to Embodiment 1-3. Configuration of Information Processing Apparatus According to Embodiment 1-4. Configuration of Terminal Device According to Embodiment <​​​​​​​​​​1-6. Output mode example (Party venue) 2. Other embodiments 2-1. Other configuration examples 2-2. Others 3. Effects according to the present disclosure 4. Hardware configuration

[0010] [1. Embodiment] [1-1. Outline of information processing according to an embodiment of the present disclosure] FIG. 1 is a diagram showing an example of information processing according to an embodiment of the present disclosure. The information processing according to the embodiment of the present disclosure is realized by a remote meeting system 1 including a remote meeting server 100 and a plurality of terminal devices 10. In FIG. 1, in order to distinguish and explain each terminal device 10, each terminal device 10 may be described as terminal device 10a, terminal device 10b, and terminal device 10c. When terminal device 10a, terminal device 10b, terminal device 10c, etc. are described without particular distinction, they are described as "terminal device 10". Also, in FIG. 1, three terminal devices 10 are illustrated, but there may be four or more terminal devices 10.

[0011] First, while referring to FIG. 1, the outline of the service regarding the remote meeting provided by the remote meeting system 1 and the definition of terms, etc. will be described. Note that detailed descriptions of points similar to general remote meeting services such as Web conferences will be omitted as appropriate. The remote meeting system 1 has a function of dividing users (participants) participating in a remote meeting (hereinafter also referred to as "meeting") into a plurality of groups. FIG. 1 shows a case where a plurality of users participating in a meeting are divided into a plurality of groups such as groups GP1 to GP4.

[0012] For example, four users U1, U2, U3, and U4 belong to group GP1, and three users U76, U11, and U23 belong to group GP2. Also, two users U98 and U12 belong to group GP3, and five users U61, U102, U543, U53, and U44 belong to group GP4. For example, in remote meeting system 1, the meeting host (administrator) divides the multiple users participating in the meeting into multiple groups. Note that the above is just an example, and in remote meeting system 1, multiple users participating in the meeting may be automatically divided into multiple groups. Furthermore, after being divided into groups, each user may be able to change the group to which they belong.

[0013] Here, the group to which each user belongs is considered the user's "first group," and all other groups to which the user belongs are considered the user's "second group." For example, for user U1, group GP1 is the first group, and the other groups GP2, GP3, GP4, etc. are the second group. Similarly, for user U76, group GP2 is the first group, and the other groups GP1, GP3, GP4, etc. are the second group.

[0014] Thus, the first and second groups are relative concepts; what is the first group for a given user is the second group for all users who do not belong to the same group as that user. For example, group GP1 is the first group for users U1, U2, U3, and U4 who belong to group GP1, and the second group for all users who do not belong to group GP1. Also, for example, if user U1 changes groups from group GP1 to group GP2 (hereinafter also referred to as "moving"), group GP2 becomes the first group for user U1, and group GP1 becomes the second group.

[0015] Furthermore, the following describes how to estimate whether a user's participation status in a group is that of a "participant" or a "bystander." Here, "participant" corresponds to the first state, which is, for example, actively participating in the group conversation, while "bystander" corresponds to the second state, which is any state other than the first state, i.e., not being a participant. Note that "participant" and "bystander" are merely examples of participation statuses, and participation status can be set arbitrarily.

[0016] Below, based on the above premises, an overview of the information processing of this application will be explained using Figure 1. Note that the step numbers such as S11-1 shown in Figure 1 are for explaining each process (codes) and do not indicate the order of processing; each process is executed as needed in accordance with the progress of the remote meeting.

[0017] Furthermore, while Figure 1 illustrates three users U1, U12, and U23 for illustrative purposes, the users participating in remote meetings provided by the remote meeting system 1 include a large number of users (e.g., 100 or 1000 people), such as users U2 and U3. In Figure 1, the terminal device 10 used by user U1 is referred to as terminal device 10a, the terminal device 10 used by user U12 is referred to as terminal device 10b, and the terminal device 10 used by user U23 is referred to as terminal device 10c. Each terminal device 10 has an application installed for participating in remote meetings provided by the remote meeting system 1 (hereinafter also referred to as the "remote meeting app"), and each user participates in the remote meeting using the remote meeting app.

[0018] In Figure 1, each terminal device 10 estimates the preferences of the user using that terminal device 10. Each terminal device 10 estimates the user's preferences based on the user's speech history, interests registered in advance by the user, etc., but the details will be described later. For example, terminal device 10a used by user U1 estimates user U1's preference as preference PF1 (step S10-1). In Figure 1, the information indicating the user's preference is shown as abstract information such as "PF1," but the information indicating the user's preference may also be information such as keywords that indicate the user's preferences (interests), such as "anime" or "Olympics." Note that the keywords are not limited to strings, but may also be abstractly represented information such as vector representations (for example, numerical vectors).

[0019] Furthermore, terminal device 10b used by user U12 estimates user U12's preferences as preference PF12 (step S10-2). Similarly, terminal device 10c used by user U23 estimates user U23's preferences as preference PF23 (step S10-3). Hereafter, when steps S10-1 to S10-3 are not distinguished, they will be collectively referred to as step S10. Steps S10-1 to S10-3 are not limited to a single execution, but are performed as needed in accordance with the progress of the remote meeting.

[0020] Furthermore, each terminal device 10 continuously sends and receives information related to remote meetings with the remote meeting server 100. Each terminal device 10 transmits information such as the user's speech and images to the remote meeting server 100 and receives information to be output about the remote meeting from the remote meeting server 100.

[0021] Communication (sending and receiving) of information related to the remote meeting, which is input and output by the terminal device 10a used by user U1, takes place between the remote meeting server 100 and the terminal device 10a used by user U1 (step S11-1). For example, the terminal device 10a sends information such as user U1's speech and images to the remote meeting server 100 and receives information to be output about the remote meeting from the remote meeting server 100. If the terminal device 10a estimates user U1's preferences, it sends information indicating user U1's preferences PF1 to the remote meeting server 100.

[0022] Furthermore, communication (sending and receiving) of information related to the remote meeting that is input and output by the terminal device 10b is performed between the remote meeting server 100 and the terminal device 10b used by user U12 (step S11-2). For example, the terminal device 10b sends information such as user U12's speech and images to the remote meeting server 100 and receives information to be output about the remote meeting from the remote meeting server 100. If the terminal device 10b estimates user U12's preferences, it sends information indicating user U12's preferences PF12 to the remote meeting server 100.

[0023] Furthermore, communication (sending and receiving) of information related to the remote meeting is performed between the remote meeting server 100 and the terminal device 10c used by user U23 (step S11-3). For example, terminal device 10c sends information such as user U23's speech and images to the remote meeting server 100 and receives information to be output about the remote meeting from the remote meeting server 100. If terminal device 10b estimates user U23's preferences, it sends information indicating user U23's preferences PF23 to the remote meeting server 100. Hereafter, when steps S11-1 to S11-3 are not distinguished, they will be collectively referred to as step S11. Step S11 is a step in which information related to the remote meeting is communicated (sent and received) between the remote meeting server 100 and each terminal device 10. Steps S11-1 to S11-3 are not limited to one time, but are executed as needed according to the progress of the remote meeting. Also, step S11 may be performed before step S10.

[0024] The remote meeting server 100 collects information received from each terminal device 10 and performs estimation processing to estimate the topic of each group and the participation status of each user (step S12). In Figure 1, the remote meeting server 100 collects information as shown in group information LT1 to LT4 and performs estimation processing. For group GP1, the remote meeting server 100 estimates the topic of group GP1 based on the content of the utterances of users U1, U2, U3, and U4 belonging to group GP1. The remote meeting server 100 estimates the topic of group GP1 to be topic GT1 based on the content of the utterances of users U1, U2, U3, and U4 belonging to group GP1. In Figure 1, the information indicating the group's topic is shown as abstract information such as "GT1," but the information indicating the group's topic may also be information such as keywords indicating the topic being discussed in the group, such as "anime" or "politics." Note that the keywords are not limited to strings, but may also be abstractly represented information such as vector representations (for example, numerical vectors).

[0025] Furthermore, the remote meeting server 100 estimates the participation status of users U1, U2, U3, and U4 belonging to group GP1. The remote meeting server 100 estimates whether each of users U1, U2, U3, and U4 is a participant or an observer. For example, for user U1, the remote meeting server 100 estimates whether user U1 is a participant or an observer based on user U1's speech and participation attitude in group GP1. In Figure 1, the remote meeting server 100 estimates that user U1 is an observer. The remote meeting server 100 also estimates that user U2 is a participant, user U3 is an observer, and user U4 is a participant. Details on the estimation of participation status will be described later.

[0026] Furthermore, the remote meeting server 100 estimates that the topic of group GP2 is topic GT2 based on the content of the utterances of users U76, U11, and U23 belonging to group GP2. The remote meeting server 100 also estimates that user U76 is an observer, user U11 is a participant, and user U23 is an observer. Similarly, the remote meeting server 100 estimates the topic of groups GP3, GP4, etc., and the participation status of the users belonging to those groups.

[0027] As a result, the remote meeting server 100 estimates the topics of each group GP1 to GP4 and the participation status of each user, as shown in the group information LT1 to LT4.

[0028] Then, the remote meeting server 100 performs a matching process to determine which second group (hereinafter also referred to as the "recommended group") to recommend to the user, based on the similarity between each user's preferences and the topics of the second group (other than the first group to which the user belongs) (step S13). In Figure 1, the remote meeting server 100 performs the matching process only for users whose participation status is "bystander". The remote meeting server 100 calculates the similarity between keywords indicating the user's preferences (also referred to as "participant keywords") and keywords indicating the group's topics (also referred to as "group keywords"). For example, the remote meeting server 100 vectorizes each keyword and calculates the cosine similarity between the vectors of each keyword as the similarity between keywords. For example, the remote meeting server 100 converts keywords into vectors using any model (vector transformation model) such as Word2Vec or BoW (Bag of words).

[0029] In Figure 1, the remote meeting server 100 calculates the similarity between the preferences PF1 of user U1, who is an observer, and the topics GT2, GT3, and GT4 of groups GP2, GP3, and GP4, excluding group GP1 to which user U1 belongs. For example, the remote meeting server 100 calculates the similarity between user U1 and group GP2 by calculating the cosine similarity between the participant keyword vector shown in user U1's preferences PF1 and the group keyword vector shown in group GP2's topic GT2. Similarly, the remote meeting server 100 calculates the similarity between user U1 and groups GP3 and GP4 by calculating the cosine similarity between user U1's participant keyword vector and the group keyword vector. The remote meeting server 100 then determines the group with the highest calculated similarity to be the recommended group for user U1. For example, the remote meeting server 100 determines group GP3 of topic GT3, which has the highest similarity to user U1's preferences PF1, to be the recommended group for user U1.

[0030] Furthermore, the remote meeting server 100 calculates the similarity between the preferences PF23 of the observer user U23 and the topics GT1, GT3, and GT4 of groups GP1, GP3, and GP4, excluding group GP2 to which user U23 belongs. The remote meeting server 100 then determines the group with the highest calculated similarity to be the recommended group for user U23. For example, the remote meeting server 100 determines group GP1 of topic GT1, which has the highest similarity to user U23's preferences PF23, to be the recommended group for user U23.

[0031] The remote meeting server 100 similarly determines recommended groups for other bystander users U3, U76, U543, U53, U44, etc. However, the remote meeting server 100 does not have to determine recommended groups for the participants users U2, U4, U11, U98, U12, U61, U102, etc., or it may. Furthermore, the remote meeting server 100 does not have to send the notification described later to users who are participants, or it may do so. For example, if the remote meeting server 100 sends the notification described later to users who are participants, it may do so in a less conspicuous manner than to bystanders (for example, by making the display smaller than that for bystanders), or it may do so in the same way as bystanders.

[0032] The remote meeting server 100 notifies each user of the recommended group determined by the matching process. The remote meeting server 100 also notifies users who are observing of the recommended group for them.

[0033] For example, the remote meeting server 100 notifies user U1 of group GP3, which is a recommended group for user U1 (step S14-1). For example, the remote meeting server 100 notifies user U1 of group GP3 by sending information indicating group GP3 to the terminal device 10a used by user U1. For example, the remote meeting server 100 notifies user U1 of group GP3 by sending information to the terminal device 10a in order to display screen IM1 containing information about group GP3 as shown in Figure 2, but details of Figure 2 will be described later.

[0034] Furthermore, the remote meeting server 100 notifies user U23 of group GP1, which is a recommended group for user U23 (step S14-3). For example, the remote meeting server 100 notifies user U23 of group GP1 by sending information indicating group GP1 to the terminal device 10c used by user U23. For example, the remote meeting server 100 notifies user U23 of group GP1 by sending information to the terminal device 10c in order to display a screen containing information about group GP1.

[0035] In the example shown in Figure 1, since user U12 is a participant at that time, the terminal device 10b used by user U12 is not notified of the recommended group. However, if user U12 changes from a participant to an observer, the remote meeting server 100 may perform a matching process for user U12 and notify user U12 of the recommended group (step S14-2). Hereafter, when steps S14-1 to S14-3 are not distinguished, they will be collectively referred to as step S11. Steps S14-1 to S14-3 are executed as needed when the conditions are met in accordance with the progress of the remote meeting. Step S14 may also be performed together with step S11.

[0036] This allows the remote meeting system 1 to notify participants joining a remote meeting of appropriate information. From here, using Figures 2 to 4, we will explain examples of how the terminal device 10 displays information depending on whether or not a recommended group notification is provided.

[0037] [1-1-1. Example of display with notifications for recommended groups] First, we will explain an example of a display showing notifications for recommended groups using Figure 2. Figure 2 shows an example of a display for bystanders. In Figure 2, we show an example of the display on terminal device 10a of user U1, who is a bystander in Figure 1.

[0038] The display 15 of the terminal device 10 shown in Figure 2 displays screen IM1, which contains information about group GP3, the recommended group for user U1. Screen IM1 primarily displays information about group GP1, which is the group to which user U1 belongs (the first group). The information about group GP1 in screen IM1 includes images of users participating in group GP1. In Figure 2, area AR1 displays images of users participating in group GP1, and area AR3 displays images of users participating in group GP3. Thus, the terminal device 10 displays images of participants in group GP1, the first group, as well as images of participants in group GP3, the recommended group (the second group).

[0039] Furthermore, as shown in Figure 2, the first area AR1 where group GP1 is displayed is larger than the second area AR3 where group GP3 is displayed. In other words, on the terminal device 10, the images of the participants of the first group, group GP1, are displayed larger than the images of the participants of the recommended group (second group), group GP3. Thus, on the terminal device 10, the images of the participants of the recommended group (second group), group GP3, are displayed in a second area that is smaller than the first area where the images of the participants of the first group, group GP1, are displayed. This allows the remote meeting system 1 to display the first group, group GP1, and the recommended group (second group), group GP3, in an identifiable manner. For example, the remote meeting server 100 notifies the user U1 of the recommended group (second group) by sending output data for displaying screen IM1 to the terminal device 10a.

[0040] Furthermore, below the information for group GP1 in screen IM1, a chat box IN1 is placed for sending text messages to users of group GP1. To the right of the chat box IN1 in screen IM1, a group list GL is placed to display multiple groups in a list. Within the group list GL, the rectangle enclosed by a thick line indicates group GP1, to which user U1 currently belongs, and the rectangle enclosed by a dotted line indicates group GP3, which is a recommended group for user U1. If a group is selected by hovering the mouse over the group list GL, a list of participants in the selected group may be displayed in a pop-up window.

[0041] Furthermore, the notification area NT1 on screen IM1 displays information about group GP3, which is a recommended group for user U1. The information about group GP3 displayed in the notification area NT1 includes images of the users participating in group GP3. Below the information about group GP3 in the notification area NT1 is a move button BT1 for navigating to group GP3. For example, if user U1 selects the move button BT1, the group to which user U1 belongs changes from group GP1 to group GP3. Below the move button BT1 in the notification area NT1 is a chat box IN2 for sending text messages to users in group GP3.

[0042] The audio output of the terminal device 10 shown in Figure 2 will now be explained. The terminal device 10 shown in Figure 2 outputs the audio of Group GP3, which is the recommended group (second group) for user U1, superimposed on the audio of Group GP1, which is the first group of user U1.

[0043] Furthermore, the terminal device 10 shown in Figure 2 superimposes the audio of group GP3 at a second volume that is lower than the first volume of group GP1 to which user U1 belongs. In this case, the remote meeting server 100 sets the parameters so that the second volume of group GP3, which is the recommended group (second group) for user U1, is lower than the first volume of group GP1, which is user U1's first group, and notifies the terminal device 10a.

[0044] Note that the display example in Figure 2 is merely one example, and any display method can be adopted for bystanders. An example of this will be explained using Figure 3. Figure 3 shows another example of a display for bystanders. Note that in Figure 3, the same reference numerals are used for points similar to those in Figure 2, and explanations are omitted as appropriate.

[0045] Terminal device 10 may display information indicating what kind of conversations are taking place in the recommended group, group GP3. In Figure 3, terminal device 10 displays the group keyword KW, which indicates what kind of conversations are taking place in group GP3, as a pop-up on the screen. The group keyword KW for group GP3, "Anime XX," is a keyword that indicates information (topics) about what kind of conversations are taking place in group GP3. Although it is shown abstractly as "Anime XX" in Figure 3, "Anime XX" is a string of characters that represents the title of a specific anime.

[0046] [1-1-2. Example of display without notifications for recommended groups] Next, we will explain an example of a display without notification for recommended groups using Figure 4. Figure 4 is a diagram showing an example of a display to a party. In Figure 4, we show an example of the display on the terminal device 10 of user U2, who is a party in Figure 1. In Figure 4, the same reference numerals are used for points that are the same as in Figure 2, and explanations are omitted as appropriate.

[0047] The display 15 of the terminal device 10 shown in Figure 4 displays screen IM2, which does not include information about recommended groups for user U2. Thus, screen IM2 differs from screen IM1 in that it does not include a notification area NT1 that notifies the user of recommended groups. In addition, on screen IM2, information about group GP1, which is the group to which user U2 belongs (first group), may be expanded and displayed up to the notification area NT1 in Figure 2. For example, the display of notification area NT1 may be collapsed and hidden, or its space may be reduced. Furthermore, if user U2 transitions from participant to observer, notification area NT1 may be displayed at the time of the transition.

[0048] [1-1-3. Background and Effects, etc.] Traditional remote meetings have presented challenges such as difficulty for multiple people to speak simultaneously, with other participants often becoming passive listeners while one person speaks, and a lack of engaging discussion when there are many participants. Dividing participants into smaller groups is also an option, but this presents the problem of not knowing what is happening in other groups. These problems are particularly pronounced in remote meetings where the agenda and flow are not predetermined, such as brainstorming sessions or informal gatherings.

[0049] Therefore, in remote meetings with multiple participants, the remote meeting system 1 uses speech recognition to estimate the topic being discussed by each participant, and ensures that each participant hears the voices of participants whose preferences and interests are similar to their own. In this case, for example, participants may be placed in different spatial locations, and the display and audio output may be performed according to their spatial relationships, but this point will be discussed later.

[0050] Furthermore, in remote meeting system 1, each participant's preferences (interests) can be registered in advance or inferred from their comments during the meeting. In remote meeting system 1, in systems where conversations take place in multiple groups, the above topics are compiled for each group, and the conversations exchanged within those groups are superimposed onto the conversations in the group the user is participating in. This allows remote meeting system 1 to achieve what is known as the cocktail party effect in remote meetings.

[0051] Furthermore, the remote meeting system 1 controls how audio is superimposed and how it appears on the screen according to the situation, such as when speaking or listening, allowing participants to easily grasp topics of interest in other groups and move between groups to join the conversation.

[0052] In this application, a "group" refers to a small meeting space virtually provided within a remote meeting in which each user (terminal device 10) participates, and corresponds to a breakout room in the web conferencing system disclosed in Non-Patent Document 1. While a group often includes multiple users, a group may also include only one user.

[0053] [1-1-4. User participation status] From here, we will explain examples of various types of information and processing. First, we will explain the user's participation state. A user participating in a remote meeting can be in two states: one where they are actively speaking and interacting (State 1), and another where they are observing and listening to others (State 2). A single participant will transition between these two states while participating in the meeting.

[0054] For example, the remote meeting system 1 can estimate whether a user is participating as a party (first state) based on the following information. For example, the remote meeting system 1 can estimate based on the frequency with which the user (the user) speaks. In this case, the remote meeting system 1 estimates that a user whose frequency of speaking in a group over a given period is above a threshold is a party to that group. The remote meeting system 1 also estimates that a user whose frequency of speaking in a group over a given period is below a threshold is an observer of that group. Note that since some people are naturally less talkative, the remote meeting system 1 may change the threshold for each user.

[0055] Furthermore, the remote meeting system 1 may estimate whether the user is a participant or an observer based on information such as whether the microphone remains muted, whether other people continue to comment on topics the user (the user) has mentioned, whether the user is nodding or acknowledging what others are saying, and whether other people are speaking towards the user on the screen (for example, whether their gaze is directed towards that part).

[0056] The above is merely an example, and the user's participation status may be determined appropriately using various information. For example, the user's participation status may be static, such as being pre-configured. The remote meeting system 1 may also generate a model to estimate the user's participation status using machine learning based on various information, and use the generated model to estimate the user's participation status. Furthermore, approval from existing participants may be required to join each group.

[0057] [1-1-5. Group to present] Next, we will explain an example of how to select (determine) the groups to present (notify) to the user. Remote meeting system 1 determines which group to recommend to each user by matching the content of the conversations in each group with the results of the preference estimation for each user.

[0058] As will be explained in more detail later using flowcharts, etc., in the remote meeting system 1, the user's speech is converted into text by speech recognition for each terminal device 10. The remote meeting system 1 then estimates the preferences of each terminal user. Information such as the user's past speech history (history of speech recognition results), interests registered by the user in advance, activity history on devices such as PCs and smartphones, such as website browsing history similar to web advertisements, and real-world activity history such as jogging and aerobics (also simply called "aerobics") measured by smart trackers using vital sensors, etc., is used to estimate preferences.

[0059] The user's speech history consists of the results of speech recognition of the user's speech within the system, as well as a history of keywords extracted from that recognition. The behavioral history includes records of group switching (movement) and the topics discussed within those groups.

[0060] Furthermore, users can register their interests in advance as keywords or categories, such as professional baseball, politics, jazz, or the Beatles. This can be done by selecting from categories and keywords pre-prepared by the system, or by setting their own free keywords.

[0061] Next, the remote meeting system 1 collects the text of the speech recognition results of the user utterances of the participants in each group. Then, the remote meeting system 1 estimates the topic of conversation for each group from the history of utterances for each group.

[0062] The remote meeting system 1 then calculates the similarity between users of each terminal device 10 and each group based on the preference estimation results of each terminal user and the topic estimation results of each group. Here, the method for calculating similarity can utilize document classification techniques in natural language processing. For example, the remote meeting system 1 may calculate similarity using cosine similarity, utilizing words appearing in the utterances, interests, and visited websites of users of each terminal device 10, as well as words appearing in the utterance history of each group. Alternatively, the remote meeting system 1 may use machine learning techniques such as deep learning to convert these words and sentences into vector embedding representations, build a model through learning that regressively estimates similarity using these representations, and then calculate similarity using that model. Note that the above is merely an example, and the remote meeting system 1 may calculate similarity using any method that allows for the calculation of similarity between users and each group.

[0063] The remote meeting system 1 then selects the group with the highest similarity for each user using the terminal device 10 and presents it as a recommended group. The remote meeting system 1 may also set a threshold and determine that there are no groups with similarity exceeding that threshold for a given user. If it determines that there are no recommended groups, the remote meeting system 1 does not notify the user of any recommended groups.

[0064] [1-1-6. Notification Methods] Next, we will explain examples of notification methods. First, we will explain how to present (notify) recommended groups.

[0065] Recommended groups can be presented primarily through audio, images, or a combination of both. In the case of audio presentation, the remote meeting system 1 can adjust the volume of each group's audio. The remote meeting system 1 primarily uses the audio of the user's own group, overlaying the audio of the recommended group at a lower volume so that the user can also hear the conversations of other groups. The volume of the recommended group's audio can also be individually adjusted by the user. Alternatively, the direction from which the sound is heard can be changed. The remote meeting system 1 plays the conversations of each group by overlaying the conversation of the user's own group so that it sounds from the front, and the conversation of the recommended group sounds from the right.

[0066] Furthermore, various methods of image presentation are possible by devising screen designs. As one method, the remote meeting system 1 displays the camera images of the participants in the user's own group in the center of the screen, and the camera images of the participants in the recommended group in a smaller size on the side. In addition, the display of the recommended group can be folded and hidden at times when it is not suitable for recommendation, in relation to the presentation timing described later.

[0067] The second method involves the remote meeting system 1, which provides multiple conversation spaces in a party venue-like image. The camera images of participants in the same conversation space as the user (the user) are displayed, and the conversation space of a recommended group is displayed next to it on the screen. The remote meeting system 1 can also combine the first and second methods to display the camera images of the participants in the recommended group in a smaller size.

[0068] The above describes how to present recommended groups using audio and images, but it is also important that these two modal presentations are presented in a way that is easily recognizable to the user. For example, if the direction from which the voices are heard is changed and superimposed for each group, it is desirable that the direction from which the voices are heard corresponds to the direction in which the group is displayed on the screen. If the camera images of the participants in the user's own group as well as the camera images of the participants in the recommended group are displayed, and the audio of the former is superimposed so that it is heard from the front and the audio of the latter is heard from the right, then the remote meeting system 1 should display the face image of the latter participant to the right of the face image of the former participant on the screen. Also, if a group is displayed with a party-like image, it is desirable that the remote meeting system 1 make it so that the audio of that group is heard from the direction in which the group is talking in the party venue. More details on this point will be discussed later.

[0069] Next, we will explain the timing of suggestion group notifications. The timing of suggestion group notifications varies depending on the user's participation status and whether there are any groups to recommend to the user. An effective timing for notification is when the user is participating in a group as an observer and there are groups to recommend. Also, as mentioned above, it is effective to collapse and hide the suggestion group display when the user becomes an active participant. The extent to which users want to be shown suggested groups also depends on the user. Therefore, the remote meeting system 1 may, for example, present suggested groups when the user presses a button when they find the topic of the group they are participating in boring, or it may always display suggested groups, and these settings should be made available to the user.

[0070] [1-1-7. Group changes, etc.] Next, we will explain an example regarding group switching. In remote meeting system 1, when a recommended group is notified (presented), the decision of whether or not to switch groups is basically left to the user. For example, one could consider providing a switch button next to the display of the recommended group, allowing the user to switch groups by pressing this button.

[0071] It's also possible for the organizer (host) to configure the system to automatically move participants between groups. The timing of these moves could be based on time intervals, or when a group highly recommended for each individual participant exists. This feature would likely be effective in brainstorming sessions and parties.

[0072] Furthermore, it's conceivable to implement a feature that restricts movement when the user (themselves) is involved, for example, when another user in a group is talking to the user (themselves) while looking at them.

[0073] Furthermore, in the remote meeting system 1, various methods can be employed to determine how a participant's movement is perceived by other participants in the group. Several methods are possible for how other participants are notified when a participant moves within the group.

[0074] For example, regarding how other participants in the group will see the departure, options include not sending any notification, having the departure disappear from the participant list's camera feed (fading out), or simply notifying them of the departure with a message such as "XX has left the group." Note that the above is just one example; regarding how other participants will see the departure, the departing participant could leave a message when moving to a recommended group, and that message could be displayed. In this case, a template message could appear in the chat box upon departure, from which the participant could select a message to send to other participants.

[0075] Furthermore, various methods can be employed to determine how a moving participant appears to other participants in the group. Several methods can also be considered for notifying other participants in the group of a new participant moving in.

[0076] For example, the way in which a participant moves to a group may be displayed to other participants may be the same as when a new participant is added. For example, this could include not sending any notification, sending a notification such as "○○ has joined," notifying that the participant has moved because they are interested in the topic of this group, or sending a message such as "It seems that the participant has joined because they are interested in the topic of this group." In addition, if there are many first-time participants in a gathering such as a party, the remote meeting system 1 may also notify the profile information of the user who has moved (the user themselves).

[0077] Furthermore, various configurations can be adopted for the groups to which utterances are transmitted. The most common methods for sending messages to groups in which a user participates are voice and text (chat). Since different groups are conversing on different topics, it is generally not common to send the same message to multiple groups simultaneously, however, it is possible to switch between multiple groups when sending messages. This allows for a communication style where, for example, a user continues to exchange voice messages in their current group while sending text messages to recommended groups. Through the processes described above, the remote meeting system 1 can notify participants of a remote meeting of appropriate information.

[0078] [1-2. Configuration of the remote meeting system according to the embodiment] The remote meeting system 1 shown in Figure 5 will now be described. As shown in Figure 5, the remote meeting system 1 includes a remote meeting server 100 and a plurality of terminal devices 10. The remote meeting server 100 and each of the plurality of terminal devices 10 are connected to each other via a predetermined communication network (network N) by wired or wireless means. Figure 5 is a diagram showing an example configuration of the remote meeting system according to the embodiment. Although only three terminal devices 10 are shown in Figure 5, the remote meeting system 1 includes a number of terminal devices 10 equal to or greater than the number of users participating in the remote meeting. Furthermore, the remote meeting system 1 shown in Figure 5 may include a plurality of remote meeting servers 100.

[0079] Terminal device 10 is a device used by users participating in a remote meeting. Terminal device 10 outputs information related to the remote meeting. Terminal device 10 displays images (video) of the remote meeting and outputs audio of the remote meeting. For example, terminal device 10 sends user speech and images (video) to the remote meeting server 100 and receives audio and images (video) of the remote meeting from the remote meeting server 100.

[0080] Terminal device 10 accepts input from the user. Terminal device 10 accepts voice input from the user through speech and input from the user through operation. Terminal device 10 can be any device as long as it can realize the processing in the embodiment. Terminal device 10 can be any device as long as it has a configuration that has functions such as displaying information and outputting audio for remote meetings. For example, terminal device 10 may be a notebook PC (Personal Computer), tablet terminal, desktop PC, smartphone, smart speaker, television, mobile phone, PDA (Personal Digital Assistant), etc.

[0081] Furthermore, the terminal device 10 has speech recognition capabilities. For example, the terminal device 10 has natural language understanding (NLU) and automatic speech recognition (ASR) capabilities. The terminal device 10 may also have software modules for speech signal processing, speech recognition, speech semantic analysis, and dialogue control. For example, the terminal device 10 may transcribe the user's utterance into text and use the transcribed utterance (i.e., the textual information of the utterance) to estimate the user's speech content and preferences. The terminal device 10 may also communicate with a speech recognition server that has natural language understanding and automatic speech recognition capabilities, and obtain the transcribed utterance and information indicating the estimated speech content and preferences from the speech recognition server.

[0082] Furthermore, the remote meeting system 1 may include an administrator terminal used by the host (administrator) of the remote meeting. Note that one of the multiple terminal devices 10 may be the administrator terminal used by the administrator. For example, the administrator may use the administrator terminal 10 to invite users to a meeting or divide users participating in the meeting into groups.

[0083] The remote meeting server 100 is a computer used to provide remote meeting services to users. The remote meeting server 100 is an information processing device that, when a user is an observer, notifies the user of groups other than the group to which the user belongs that match the user's preferences. The remote meeting server 100 may also have speech recognition functions such as natural language understanding and automatic speech recognition. For example, the remote meeting server 100 may transcribe the user's speech into text and use the transcribed speech (i.e., the textual information of the speech) to estimate the user's speech content and preferences.

[0084] [1-3. Configuration of the information processing device according to the embodiment] Next, the configuration of a remote meeting server 100, which is an example of an information processing device that performs information processing according to the embodiment, will be described. Figure 6 is a diagram showing an example of the configuration of a remote meeting server according to the embodiment of this disclosure.

[0085] As shown in Figure 6, the remote meeting server 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The remote meeting server 100 may also have an input unit (e.g., a keyboard or mouse) for receiving various operations from the administrator of the remote meeting server 100, and a display unit (e.g., a liquid crystal display) for displaying various information.

[0086] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N (see Figure 5) by wire or wireless connection and transmits and receives information with other information processing devices such as the terminal device 10. The communication unit 110 may also transmit and receive information with a user terminal (not shown) used by the user.

[0087] The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. As shown in Figure 6, the storage unit 120 according to this embodiment has a user information storage unit 121 and a group information storage unit 122.

[0088] The user information storage unit 121 according to this embodiment stores various information about the user. For example, the user information storage unit 121 stores information about users participating in a remote meeting. The user information storage unit 121 stores information about each user's interests and preferences. The user information storage unit 121 stores preference estimation results for each user in association with the user. The user information storage unit 121 stores user information in association with information that identifies each user (such as a user ID).

[0089] The group information storage unit 122 according to this embodiment stores various information about a group. The group information storage unit 122 stores various information about multiple groups, each corresponding to a multiple group in a remote meeting. Figure 7 is a diagram showing an example of the group information storage unit according to this disclosure. Figure 7 shows an example of the group information storage unit 122 according to this embodiment. In the example shown in Figure 7, the group information storage unit 122 includes items such as "group ID", "topic", "participants", and "status".

[0090] The "Group ID" indicates identification information used to identify a group. The "Topic" indicates the topic of discussion within the corresponding group. For example, the "Topic" indicates a topic estimated based on the content of the group's dialogue. Figure 6 shows an example where conceptual information such as "GT1" is stored in the "Topic," but in reality, specific information such as keywords indicating the group's topic, such as "anime" or "Olympics," is stored.

[0091] "Participants" refers to the participants in the group. For example, "Participants" stores information (such as a user ID) that identifies the participants (users) belonging to the group. "Status" indicates the status of each participant. Figure 7 shows a case where either "Party Member" or "Bystander" is stored in "Status," but the status can be set to any desired value, not just "Party Member" or "Bystander."

[0092] In the example shown in Figure 7, the group identified by group ID "GP1" (Group GP1) includes participants such as user identified by user ID "U1" (User U1), user identified by user ID "U2" (User U2), user identified by user ID "U3" (User U3), and user identified by user ID "U4" (User U4). It also indicates that the topic of Group GP1 is topic GT1. In Group GP1, at that time, users U2 and U4 are parties involved, while users U1 and U3 are bystanders.

[0093] Furthermore, the group information storage unit 122 may store various types of information depending on the purpose, not limited to those mentioned above.

[0094] Furthermore, the storage unit 120 may store various other types of information besides those mentioned above. For example, the storage unit 120 may store various types of information related to remote meetings. The storage unit 120 may store various types of data for providing output data to the terminal device 10. For example, the storage unit 120 may store various types of information used to generate information displayed on the terminal device 10 of users participating in remote meetings. For example, the storage unit 120 may store information related to content displayed by applications (such as remote meeting applications) installed on the terminal device 10. For example, the storage unit 120 may store information related to content displayed by remote meeting applications. Note that the above are merely examples, and the storage unit 120 may store various types of information used to provide remote meeting services to users.

[0095] Returning to Figure 6, let's continue the explanation. The control unit 130 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (for example, the information processing program related to this disclosure) stored inside the remote meeting server 100 using RAM (Random Access Memory) as the working area. Alternatively, the control unit 130 can be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0096] As shown in Figure 6, the control unit 130 includes an acquisition unit 131, a data holding unit 132, a group management unit 133, a topic matching unit 134, an output generation unit 135, and a notification unit 136, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 6, and other configurations are also possible as long as they perform the information processing described later. Also, the connection relationships of the various processing units in the control unit 130 are not limited to the connection relationships shown in Figure 6, and other connection relationships are also possible.

[0097] The acquisition unit 131 acquires various types of information. The acquisition unit 131 acquires various types of information from an external information processing device. The acquisition unit 131 acquires various types of information from the terminal device 10. The acquisition unit 131 acquires various types of information from the terminal device 10 based on information detected by the operation unit 16 (described later in Figure 7) of the terminal device 10. The acquisition unit 131 acquires various types of information from the terminal device 10 based on information detected by the voice input unit 12 (described later in Figure 7).

[0098] The acquisition unit 131 acquires various information from the storage unit 120. The acquisition unit 131 acquires various information from the user information storage unit 121 and the group information storage unit 122. The acquisition unit 131 acquires various information calculated by the data holding unit 132. The acquisition unit 131 acquires various information determined by the group management unit 133.

[0099] The acquisition unit 131 acquires information related to the remote meeting. The acquisition unit 131 acquires information such as the user's speech and images. The acquisition unit 131 receives information such as the user's speech and images from the terminal device 10. The acquisition unit 131 receives information from the terminal device 10 that indicates the results of the user's preference estimation. The acquisition unit 131 receives information such as the user's image and voice, speech recognition results, and preference estimation results from the terminal device 10.

[0100] The data storage unit 132 performs data storage processing. The data storage unit 132 stores information such as images, audio, speech recognition results, and preference estimation results transmitted from each terminal device 10. The data storage unit 132 stores information such as images, audio, speech recognition results, and preference estimation results transmitted from each terminal device 10 in the storage unit 120.

[0101] The group management unit 133 performs processes related to group management. The group management unit 133 manages the groups in which users of each terminal device 10 are members. For each group, the group management unit 133 uses the speech recognition results of the participants to estimate the topics of conversation taking place in that group.

[0102] The group management unit 133 functions as an estimation unit that performs estimation processing. The group management unit 133 estimates the participation status of each user participating in a group. The group management unit 133 assumes that multiple participants in a remote meeting are divided into multiple groups, and that each group can converse within its own group, and estimates the topic of conversation within the group from the statements of the participants belonging to that group. The group management unit 133 estimates the participation status of one participant in the first group. The group management unit 133 estimates whether the state of one participant is a first state, which is an actively participating state, or a second state, which is a state other than the first state.

[0103] The topic matching unit 134 performs processing related to topic matching. The topic matching unit 134 performs matching processing using the speech recognition results and preference estimation results from each terminal device 10 and the topic estimation results for each group estimated by the group management unit 133. In addition to the groups in which the users of each terminal device 10 are participating, the topic matching unit 134 matches groups whose topics are similar to the preferences of those users.

[0104] The topic matching unit 134 performs matching based on the similarity between keywords indicating the user's preferences and keywords indicating topics in each group other than the group to which the user belongs (the first group) (second group). The topic matching unit 134 determines whether the comparison result between the keywords indicating the user's preferences and the keywords indicating topics in each second group satisfies the similarity conditions. The topic matching unit 134 determines the group with the highest similarity among the multiple second groups to be the group to notify the user of (notification group).

[0105] The topic matching unit 134 may also use a threshold. In this case, the topic matching unit 134 determines whether the similarity between the keyword indicating the user's preferences and the keyword indicating the topic of each second group is above a predetermined threshold. The topic matching unit 134 then determines which of the multiple second groups has a similarity above the predetermined threshold to be the group to be notified to the user (notification group). For example, if there are multiple groups among the multiple second groups whose similarity is above the predetermined threshold, the topic matching unit 134 determines which group has the greatest similarity to be the group to be notified to the user (notification group).

[0106] The output generation unit 135 generates various types of information. The output generation unit 135 generates various types of information based on information from external information processing devices and information stored in the storage unit 120. The output generation unit 135 generates various types of information based on information from other information processing devices such as the terminal device 10. The output generation unit 135 generates various types of information based on information stored in the user information storage unit 121 and the group information storage unit 122. The output generation unit 135 generates various types of information to be displayed on the terminal device 10 based on information determined by the group management unit 133.

[0107] The output generation unit 135 prepares the information in the required output format, including the audio and images of the groups in which each terminal device 10 user is participating, as well as the audio and images of the groups matched by the topic matching unit 134. For example, the output generation unit 135 adjusts various parameters according to the group to which the user belongs and the user's participation status. For example, the output generation unit 135 adjusts parameters according to whether the user is a participant or an observer. For example, the output generation unit 135 may generate information to be provided to the terminal device 10 using various technologies related to remote meetings such as web conferencing.

[0108] The output generation unit 135 generates output data to be provided to the terminal device 10 using the adjusted parameters. The output generation unit 135 generates output data to be used by the terminal device 10 to output information about the remote meeting. For example, the output generation unit 135 generates output data that includes parameters such as the volume of each group and the placement and size of images for each group.

[0109] The output generation unit 135 generates output data that superimposes the voices of the participants of the second group onto the voices of the participants of the first group. The output generation unit 135 generates output data that superimposes the voices of the participants of the second group at a second volume that is lower than the first volume of the first group. The output generation unit 135 generates output data that displays the images of the participants of the second group along with the images of the participants of the first group. The output generation unit 135 generates output data that displays the first group and the second group in a distinguishable manner. The output generation unit 135 generates output data that displays the images of the participants of the second group in a second area that is smaller than the first area that displays the images of the participants of the first group.

[0110] When the output generation unit 135 notifies the audio and images of the participants in the second group, it generates output data that notifies the correspondence between the audio and images of the participants in the second group in a manner that allows recognition. The output generation unit 135 generates output data that notifies the audio of the participants in the second group from a direction corresponding to the relationship between the display positions of the first group and the second group.

[0111] Furthermore, the output generation unit 135 may generate a display screen (content) to be displayed on the terminal device 10 as output data. For example, the output generation unit 135 may generate a screen (content) to be provided to the terminal device 10 using various technologies such as Java (registered trademark) as appropriate. The output generation unit 135 may also generate a screen (content) to be provided to the terminal device 10 based on the format of CSS, JavaScript (registered trademark), or HTML. In addition, for example, the output generation unit 135 may generate a screen (content) in various formats such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or PNG (Portable Network Graphics).

[0112] The notification unit 136 performs processing related to notifying the user. The notification unit 136 transmits information to be notified to the terminal device 10. The notification unit 136 transmits the information generated by the output generation unit 135 to the terminal device 10. The notification unit 136 transmits the output data generated by the output generation unit 135 to the terminal device 10. The notification unit 136 transmits information indicating the matched group by the topic matching unit 134 to the terminal device 10. If the topic matching unit 134 determines a second group for the user and that user is an observer, the notification unit 136 determines that the conditions are met and notifies the user.

[0113] The notification unit 136 notifies a participant of the second group if the topic of the second group (a group other than the first group to which the participant belongs) satisfies the conditions regarding similarity between the topic of the second group and the participant's preferences. The notification unit 136 changes its notification response regarding the second group according to the participant's participation status in the first group, as estimated by the group management unit 133.

[0114] The notification unit 136 changes the notification method of the second group to a participant depending on whether the participant is in the first state or the second state. The notification unit 136 changes the timing of the notification of the second group to a participant depending on whether the participant is in the first state or the second state. The notification unit 136 changes the display method of the second group to a participant depending on whether the participant is in the first state or the second state.

[0115] The notification unit 136 notifies a participant of the second group if that participant is in the second state. The notification unit 136 notifies a participant of the second group at the moment the participant's participation state transitions from the first state to the second state. The notification unit 136 superimposes the voices of the participants in the second group onto the voices of the participants in the first group. The notification unit 136 superimposes the voices of the participants in the second group at a second volume that is lower than the first volume of the first group.

[0116] The notification unit 136 displays images of the participants of the second group along with images of the participants of the first group. The notification unit 136 displays the first group and the second group in a way that allows them to be distinguished. The notification unit 136 displays images of the participants of the second group in a second area that is smaller than the first area that displays images of the participants of the first group.

[0117] When the notification unit 136 notifies the audio and images of participants in the second group, it notifies in a manner that allows the correspondence between the audio and images of the participants in the second group to be recognized. The notification unit 136 notifies in a manner that the audio of the participants in the second group is output from a direction corresponding to the relationship between the display positions of the first group and the second group.

[0118] The notification unit 136 notifies a participant of the second group if the group keyword, which is a keyword indicating the topic of the second group, and the participant keyword, which is a keyword indicating the preferences of one participant, satisfy the conditions regarding similarity. The notification unit 136 notifies a participant of the second group if the comparison result between the group keyword of the second group and the participant keyword of one participant satisfies the conditions regarding similarity. The notification unit 136 notifies a participant of the second group if the similarity between the group keyword of the second group and the participant keyword of one participant is above a predetermined threshold.

[0119] [1-4. Configuration of the terminal device according to the embodiment] Next, the configuration of a terminal device 10, which is an example of an information processing apparatus that performs information processing according to the embodiment, will be described. Figure 8 is a diagram showing an example of the configuration of a terminal device according to the embodiment of this disclosure.

[0120] As shown in Figure 8, the terminal device 10 includes a communication unit 11, an audio input unit 12, an audio output unit 13, a camera 14, a display 15, an operation unit 16, a storage unit 17, and a control unit 18.

[0121] The communication unit 11 is implemented, for example, by a NIC or communication circuit. The communication unit 11 is connected to a predetermined communication network by wire or wireless and transmits and receives information with an external information processing device. For example, the communication unit 11 is connected to a predetermined communication network by wire or wireless and transmits and receives information with a remote meeting server 100.

[0122] The voice input unit 12 is, for example, a microphone, and detects sound. For example, the voice input unit 12 detects the user's speech. The voice input unit 12 can have any configuration as long as it can detect the user's speech information necessary for processing.

[0123] The audio output unit 13 is implemented by a speaker that outputs sound and is an output device for outputting various information as sound. The audio output unit 13 outputs content provided by the remote meeting server 100 as sound. For example, the audio output unit 13 outputs sound corresponding to the information displayed on the display 15. The terminal device 10 performs audio input and output using the audio input unit 12 and the audio output unit 13.

[0124] Camera 14 has an image sensor that detects images. Camera 14 takes pictures of users participating in a remote meeting. For example, if the terminal device 10 is a laptop computer, camera 14 may be built into the terminal device 10 and positioned at the top of the display 15. Alternatively, for example, if it is a smartphone, camera 14 may be a built-in front-facing camera in the terminal device 10.

[0125] The display 15 is a display screen for a tablet device or the like, realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device (display unit) for displaying various information.

[0126] Display 15 displays various information about the remote meeting the user is participating in. Display 15 displays content. Display 15 displays various information received from the remote meeting server 100. Display 15 outputs information about the remote meeting received from the remote meeting server 100. Display 15 displays information about the group the user belongs to. The terminal device 10 performs image input and output using the camera 14 and the display 15.

[0127] The control unit 16 accepts various user inputs. In the example shown in Figure 8, the control unit 16 is a keyboard, mouse, etc. Alternatively, the control unit 16 may have a touch panel that can perform functions equivalent to a keyboard or mouse. In this case, the control unit 16 accepts various operations from the user via the display screen through the functions of the touch panel realized by various sensors. For example, the control unit 16 accepts various operations from the user via the display 15.

[0128] For example, the operation unit 16 receives user-specified operations and other operations via the display 15 of the terminal device 10. While tablet terminals primarily use a capacitive touchscreen method for detecting user operations, other detection methods such as resistive touchscreens, surface acoustic wave methods, infrared methods, and electromagnetic induction methods may be used as long as they can detect user operations and realize the functionality of a touch panel.

[0129] The keyboard, mouse, touch panel, etc. mentioned above are merely examples, and the terminal device 10 may have a configuration that accepts (detects) various types of information as input, not limited to those mentioned above. For example, the terminal device 10 may have an eye-tracking sensor that detects the user's gaze. The eye-tracking sensor uses eye-tracking technology to detect the user's gaze direction based on the detection results of, for example, a camera 14, light sensor, motion sensor (none of which are shown) mounted on the terminal device 10. Based on the detected gaze direction, the eye-tracking sensor determines the gaze area on the screen that the user is fixated on. The eye-tracking sensor transmits gaze information, including the determined gaze area, to the remote meeting server 100. For example, the terminal device 10 may have a motion sensor that detects the user's gestures, etc. The terminal device 10 may accept user gestures via the motion sensor.

[0130] The storage unit 17 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical discs. The storage unit 17 stores, for example, various information received from the remote meeting server 100. The storage unit 17 also stores, for example, information related to applications installed on the terminal device 10 (e.g., remote meeting applications), such as programs.

[0131] The memory unit 17 stores user information. The memory unit 17 stores various types of information used to estimate user preferences. The memory unit 17 stores the user's speech history (history of speech recognition results) and behavioral history. The memory unit 17 also stores interests that the user has registered in advance. The memory unit 17 stores the history of actions taken by the user using devices such as the terminal device 10. The memory unit 17 stores the user's online activity history, such as website browsing history. The memory unit 17 also stores the user's real-world activity history, such as jogging and aerobics, using a smart tracker with vital sensors, etc.

[0132] Here, the user's speech history consists of the results of speech recognition of the user's speech within the system, and the history of keywords extracted from those results. The behavioral history includes the history of group switching (changes) and the topics discussed within those groups. Furthermore, the user registers their interests in advance, registering their usual interests as keywords or categories, such as professional baseball, politics, jazz, and the Beatles. The system may allow users to select from pre-defined categories or keywords, or it may allow users to set their own free keywords.

[0133] The memory unit 17 stores information about the speech recognition application (program) that implements the speech recognition function. For example, the terminal device 10 can perform speech recognition by launching the speech recognition application (also simply called "speech recognition"). The memory unit 17 stores various information used for speech recognition. The memory unit 17 stores information about the dictionary (speech recognition dictionary) used for the speech recognition dictionary. The memory unit 17 stores information for multiple speech recognition dictionaries.

[0134] The control unit 18 is a controller, and is implemented, for example, by a CPU or MPU executing various programs stored in a storage device such as the memory unit 17 inside the terminal device 10, using RAM as the working area. For example, these various programs include programs for information processing applications (e.g., remote meeting applications). The control unit 18 is also a controller, and is implemented, for example, by an integrated circuit such as an ASIC or FPGA.

[0135] As shown in Figure 8, the control unit 18 includes an acquisition unit 181, a speech recognition unit 182, a preference estimation unit 183, and a processing unit 184, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 18 is not limited to the configuration shown in Figure 8, and other configurations are also possible as long as they perform the information processing described later. Also, the connection relationships of each processing unit in the control unit 18 are not limited to the connection relationships shown in Figure 8, and other connection relationships are also possible.

[0136] The acquisition unit 181 acquires various types of information. For example, the acquisition unit 181 acquires various types of information from an external information processing device. For example, the acquisition unit 181 stores the acquired information in the storage unit 17. The acquisition unit 181 also acquires user operation information received by the operation unit 16.

[0137] The acquisition unit 181 acquires user speech information. The acquisition unit 181 acquires user speech information detected by the voice input unit 12.

[0138] The acquisition unit 181 receives information from the remote meeting server 100 via the communication unit 11. The acquisition unit 181 receives information provided by the remote meeting server 100. The acquisition unit 181 receives content from the remote meeting server 100.

[0139] The speech recognition unit 182 performs various processes related to speech recognition. The speech recognition unit 182 performs speech recognition processing using the information stored in the storage unit 17. The speech recognition unit 182 converts the user's utterance into text information. The speech recognition unit 182 can be realized by using existing speech semantic analysis technology.

[0140] Furthermore, the speech recognition unit 182 analyzes the content of the user's speech. The speech recognition unit 182 estimates the content of the user's speech by analyzing the user's speech using various conventional technologies as appropriate. For example, the speech recognition unit 182 analyzes the content of the user's speech using natural language understanding (NLU) and automatic speech recognition (ASR) functions.

[0141] The speech recognition unit 182 uses textual information corresponding to the user's utterance to estimate (identify) the content of the user's utterance through semantic analysis. For example, the speech recognition unit 182 estimates the content of the user's utterance corresponding to the textual information by analyzing the textual information using various conventional techniques such as syntactic analysis as appropriate.

[0142] The preference estimation unit 183 performs a process to estimate the user's preferences. The preference estimation unit 183 estimates the user's preferences using the user's behavior history. The preference estimation unit 183 estimates the user's preferences using the user's speech history. The preference estimation unit 183 estimates the user's preferences based on the user's speech. The preference estimation unit 183 estimates the user's preferences at that time using various information such as the transcribed speech content and the user's interests that have been registered in advance.

[0143] The preference estimation unit 183 estimates the user's preferences using various information stored in the memory unit 17. For example, the preference estimation unit 183 estimates the user's preferences using the user's speech history (history of speech recognition results) and behavioral history. The preference estimation unit 183 also estimates the user's preferences using interests that the user has registered in advance. The preference estimation unit 183 estimates the user's preferences using the history of actions performed by the user using a device such as the terminal device 10. The preference estimation unit 183 estimates the user's preferences using the user's internet activity history, such as homepage browsing history. Furthermore, the preference estimation unit 183 estimates the user's preferences using the user's real-world activity history, such as jogging or aerobics, recorded by a smart tracker using vital sensors, etc.

[0144] The processing unit 184 performs various processes. The processing unit 184 displays various information via the display 15. For example, the processing unit 184 controls the display on the display 15. The processing unit 184 outputs various information as audio via the audio output unit 13. For example, the processing unit 184 controls the audio output of the audio output unit 13.

[0145] The processing unit 184 outputs the information received by the acquisition unit 181. The processing unit 184 outputs the content provided by the remote meeting server 100. The processing unit 184 outputs the content received by the acquisition unit 181 via the audio output unit 13 or the display 15. The processing unit 184 displays the content via the display 15. The processing unit 184 outputs the content as audio via the audio output unit 13. The processing unit 184 functions as a notification unit that notifies the user of the second group if the topics of the second group and the user's preferences meet the similarity conditions. The processing unit 184 notifies the user of the recommended group via the audio output unit 13 or the display 15.

[0146] The processing unit 184 transmits various information to an external information processing device via the communication unit 11. The processing unit 184 transmits various information to the remote meeting server 100. The processing unit 184 transmits various information stored in the storage unit 17 to an external information processing device. The processing unit 184 transmits various information acquired by the acquisition unit 181 to the remote meeting server 100. The processing unit 184 transmits sensor information acquired by the acquisition unit 181 to the remote meeting server 100. The processing unit 184 transmits user operation information received by the operation unit 16 to the remote meeting server 100. The processing unit 184 transmits information such as speech and images of the user using the terminal device 10 to the remote meeting server 100.

[0147] Furthermore, each of the processes performed by the control unit 18 described above may be implemented using, for example, JavaScript (registered trademark). Also, if the information processing and other processes performed by the control unit 18 described above are carried out by a predetermined application, each part of the control unit 18 may be implemented by, for example, the predetermined application. For example, the information processing and other processes performed by the control unit 18 may be implemented by control information received from an external information processing device. For example, if the display processing described above is carried out by a predetermined application (for example, a remote meeting application), the control unit 18 may have, for example, an application control unit that controls a predetermined application or a dedicated application.

[0148] [1-5. Procedure for Information Processing According to the Embodiment] Next, the procedures for various information processing according to the embodiment will be explained using Figures 9 to 12.

[0149] [1-5-1. Processing Procedures Related to Information Processing Devices] First, the processing flow related to the information processing device will be explained using Figure 9. Figure 9 is a flowchart of the processing procedure of the information processing device according to the embodiment of this disclosure. Specifically, Figure 9 is a flowchart of the information processing procedure by a remote meeting server 100, which is an example of an information processing device.

[0150] As shown in Figure 9, the remote meeting server 100 estimates the topic of conversation within the group from the statements of the participants belonging to the group (step S101). The remote meeting server 100 determines whether the topic of the second group, which is a group other than the first group to which one participant belongs, satisfies the conditions regarding similarity with the interests of one participant (step S102). If the topic of the second group and the interests of one participant satisfy the conditions regarding similarity (step S102: Yes), the remote meeting server 100 notifies one participant of the second group (step S103).

[0151] On the other hand, if the remote meeting server 100 finds that the topic of the second group and the interest of one of the participants do not satisfy the similarity conditions (step S102: No), it terminates processing without notifying the participant of the second group. However, even if the topic of the second group and the interest of one of the participants satisfy the similarity conditions, the remote meeting server 100 may terminate processing without notifying the participant of the second group if the participant is a party to the first group to which it belongs.

[0152] [1-5-2. Procedures for processing terminal devices] Next, the processing on the terminal device 10 side will be explained using Figure 10. Figure 10 is a flowchart of the processing procedure in a terminal device according to an embodiment of this disclosure. Figure 10 is a flowchart of an example of processing when the terminal device 10 provides a service related to remote meetings.

[0153] In Figure 10, the terminal device 10 holds audio image data in a buffer (step S201). The terminal device 10 holds audio image data, including user voice data and image data, in a buffer.

[0154] Then, the terminal device 10 performs speech recognition processing on the voice data (step S202). The terminal device 10 determines whether or not the end of the voice has been detected (step S203). Note that the end of the voice can be detected using various conventional techniques as appropriate, and a detailed explanation is omitted.

[0155] If the end of speech is detected (step S203: Yes), the terminal device 10 uses the speech recognition result to estimate the topic (step S204). For example, the terminal device 10 uses the speech recognition result to estimate the topic (preferences) of the user's speech based on the content of the user's speech. For example, the terminal device 10 extracts keywords from the text information obtained by transcribing the user's speech and estimates the extracted keywords as the topic of the user's speech, that is, the user's preferences at that time.

[0156] Then, the terminal device 10 outputs the results to a buffer (step S205). For example, the terminal device 10 adds information indicating the estimated preferences (preference information) to the buffer.

[0157] Then, the terminal device 10 outputs the data in the buffer to the server at regular intervals (step S206). The terminal device 10 sends the data held in the buffer to the remote meeting server 100 at regular intervals. For example, if there is preference information in the buffer, the terminal device 10 sends the audio image data and preference information in the buffer to the remote meeting server 100.

[0158] On the other hand, if the end of the audio is not detected (step S203: No), the terminal device 10 performs the processing in step S206 without estimating the topic of the user's speech (user preference). For example, if there is no preference information in the buffer, the terminal device 10 sends the audio image data in the buffer to the remote meeting server 100.

[0159] The above is merely an example, and the terminal device 10 may send information to the remote meeting server 100 at any time if it is capable of providing services related to remote meetings.

[0160] [1-5-3. Procedures for processing related to the remote meeting server] Next, the processing on the remote meeting server 100 side will be explained using Figure 11. Figure 11 is a flowchart of the processing procedure of the remote meeting server according to the embodiment of this disclosure. Figure 11 is a flowchart of an example of processing when the remote meeting server 100 provides services related to remote meetings.

[0161] In Figure 11, the remote meeting server 100 receives data from each terminal and stores it in the data storage unit (step S301). For example, the remote meeting server 100 receives data from each terminal device 10 and stores the received data in the storage unit 120 using the data storage unit 132.

[0162] If a certain amount of time has elapsed (Step S302: Yes), the remote meeting server 100 checks the recommended group for each terminal (Step S303). For example, the remote meeting server 100 checks the recommended group for users of each terminal device 10 at predetermined intervals (e.g., a few seconds, tens of seconds, etc.).

[0163] The remote meeting server 100 then estimates the user participation status of each terminal (step S304) and branches the processing according to the estimation result.

[0164] If the remote meeting server 100 estimates that the user's participation status is that of an observer (step S304: observer), it updates the output generation parameters that take into account the recommended group (step S305). For example, the remote meeting server 100 updates the output generation parameters used to generate output data to be sent to the user's terminal device 10 to reflect the content that takes into account the recommended group for that user. Then, the remote meeting server 100 executes the process in step S306.

[0165] On the other hand, if the remote meeting server 100 estimates that the user's participation status indicates that they are a party (step S304: Party), it executes the process in step S306 without performing the process in step S305. In other words, if the remote meeting server 100 estimates that the user's participation status indicates that they are a party, it executes the process in step S306.

[0166] The remote meeting server 100 generates output data for each terminal according to the output generation parameters (step S306). The remote meeting server 100 generates output data to be provided to the user's terminal device 10 depending on whether or not there is a recommended group for that user.

[0167] For example, if the output generation parameters of the remote meeting server 100 include information about recommended groups, it generates output data to send to the user's terminal device 10 that takes into account the recommended groups for that user. In this case, the remote meeting server 100 generates output data so that the recommended groups for that user are displayed on the user's terminal device 10, as shown in Figure 2, for example.

[0168] Furthermore, for example, if the output generation parameters of the remote meeting server 100 do not include information about recommended groups, it will generate output data to send to the user's terminal device 10 without considering the recommended groups for that user. In this case, the remote meeting server 100 will generate output data in such a way that the recommended groups for that user are not displayed on the user's terminal device 10, as shown in Figure 4, for example.

[0169] Furthermore, if a certain amount of time has not elapsed (step S302: No), the remote meeting server 100 executes the process in step S306 without performing the processes in S303 to S305. In this case, the remote meeting server 100 generates output data to be sent to the user's terminal device 10, for example, without taking into account the recommended group for that user.

[0170] Then, the remote meeting server 100 transmits data to each terminal (step S307). The remote meeting server 100 transmits the output data generated for each user to the terminal device 10 used by that user.

[0171] [1-5-4. Matching Process Procedure] Next, the matching process in the remote meeting system 1 will be explained using Figure 12. Figure 12 is a flowchart of the matching process procedure. The matching process shown in Figure 12 corresponds, for example, to steps S202 to S204 in Figure 10 and steps S301 to S303 in Figure 11. In the following explanation, the case in which the remote meeting system 1 performs the processing will be described as an example, but the process shown in Figure 12 may be performed by any of the devices, such as the remote meeting server 100 and the terminal device 10, depending on the device configuration included in the remote meeting system 1.

[0172] In Figure 12, the remote meeting system 1 converts speech into text using speech recognition (step S401). For example, the remote meeting system 1 converts the user's speech into text information using speech recognition.

[0173] Then, the remote meeting system 1 estimates the user's preferences (step S402). For example, the remote meeting system 1 extracts keywords from the text information obtained by transcribing the user's speech, and estimates that the extracted keywords represent the user's preferences.

[0174] The remote meeting system 1 then aggregates the speech recognition results of each group participant (step S403). For example, the remote meeting system 1 collects text information of the participants' (users') speech for each group.

[0175] Then, the remote meeting system 1 estimates the topic from the speech history of each group (step S404). For example, the remote meeting system 1 extracts keywords from the text information of the speech history of each group and estimates that the extracted keywords are the topic of that group.

[0176] The remote meeting system 1 then calculates the similarity between each user's preference estimation result and the topics of each group (step S405). For example, the remote meeting system 1 calculates the similarity between each user's preferences and the topics of groups other than the group to which the user belongs (group 1) (group 2).

[0177] The remote meeting system 1 then determines the group with the highest similarity to each user (step S406). For example, for each user, the remote meeting system 1 determines the group with the highest similarity (second group) as the recommended group.

[0178] [1-6. Examples of Output Formats (Party Venue)] The output format for remote meetings, such as notifications for recommended groups, is not limited to the examples shown in Figures 2 to 4, but may vary. For example, the output format may reflect the virtual location of each group or user. In this regard, an example of the display and audio output format will be explained using the party venue-style display shown in Figure 13. Note that since Figure 13 explains the display and audio output format, the explanation of the matching process and other processes that determine recommended groups, which are prerequisites, will be omitted. Also, in Figure 13, explanations of the same points as in Figures 2 to 4 will be omitted as appropriate. Figure 13 is a diagram showing an example of a display format.

[0179] Figure 13 shows an example of the display on terminal device 10 for user U22, who is an observer in group GP21. The display 15 of terminal device 10 shown in Figure 13 shows screen IM21, which includes a virtual party venue PV21 that reflects the virtual location of each group and each user, and a notification area NT21 that notifies user U22 of recommended groups.

[0180] For example, in user U22's terminal device 10, icons IC21 to IC23 corresponding to users U21 to U23 of group GP21 are placed in the virtual party venue PV21. Specifically, icon IC22 corresponding to user U22 is placed below the table in area AR21 where group GP21 is located. In addition, icon IC21 corresponding to user U21 is placed diagonally to the left and in front of icon IC22, and icon IC23 corresponding to user U23 is placed diagonally to the right and in front of icon IC22.

[0181] Furthermore, in the virtual party venue PV21, group GP25, which is the recommended group (second group) for user U22, is placed in a table within area AR25, located to the left of area AR21 where group GP21 is placed. Specifically, icons IC51 and IC52, corresponding to the two users belonging to group GP25, are placed in the table to the left of the table where group GP21 is placed.

[0182] Furthermore, the notification area NT21 on screen IM21 displays information about group GP25, which is a recommended group for user U22. The information about group GP25 displayed in the notification area NT21 includes images of the users participating in group GP25. Below the information about group GP25 in the notification area NT21 is a move button BT21 for navigating to group GP25. For example, if user U22 selects the move button BT21, the group to which user U22 belongs changes from group GP21 to group GP25. Below the move button BT21 in the notification area NT21 is a chat box IN21 for sending text messages to users in group GP25.

[0183] The audio output in the terminal device 10 shown in Figure 13 will now be explained. The terminal device 10 shown in Figure 13 outputs the audio of group GP25, which is the recommended group (second group) for user U22, by superimposing it onto the audio of group GP21, which is the first group of user U22.

[0184] Terminal device 10 outputs the audio of the participants of the second group from a direction corresponding to the relationship between the display positions of the first and second groups. In the terminal device 10 shown in Figure 13, since group GP25 is located to the left of user U22 of group GP21, terminal device 10 outputs the audio of group GP25 from the left. For example, terminal device 10 adjusts the volume of the audio of group GP25 so that the output volume to the left is greater than the output volume to the right, and outputs the audio of group GP25. In this case, the remote meeting server 100 sets the parameters for the audio of group GP25 so that the output volume to the left is greater than the output volume to the right, and notifies terminal device 10a.

[0185] For example, the stereo function of the terminal device 10 may be used to adjust and output the volume in the left and right directions, so that the voices of the participants in the second group are output from a direction corresponding to the relationship between the display positions of the first group and the second group. For example, if the terminal device 10 has speakers on both the left and right sides, the voice of group GP25 may be output only from the left speaker. Note that the above is merely an example, and when the remote meeting system 1 notifies the voices and images of the participants in the second group, it may notify in any manner that allows the correspondence between the voices and images of the participants in the second group to be recognized. For example, it may be a representation in a virtual space such as a game using VR (Virtual Reality) goggles. In this case, for example, the terminal device 10 may be VR goggles, and the terminal device 10 may display a virtual space such as a game and output the voice in a positional relationship corresponding to the displayed virtual space.

[0186] Furthermore, if user U21 is a party in group GP21, the terminal device 10 of user U21 will display a screen that does not include information about recommended groups for user U21 or the notification area NT21. For example, the terminal device 10 of user U21 may display a virtual party venue PV21 in which only icons IC21 to IC23 corresponding to users U21 to U23 of group GP21 are placed.

[0187] [2. Other Embodiments] The processes according to each of the embodiments described above may be carried out in various different forms (modifications) other than the embodiments and modifications described above.

[0188] [2-1. Other Configuration Examples] In the remote meeting system 1 example described above, the terminal device 10 performs speech recognition and preference estimation, and the remote meeting server 100 performs group topic estimation and matching processing. However, any configuration can be adopted for the division of functions in the remote meeting system 1.

[0189] For example, in the remote meeting system 1, the remote meeting server 100 may perform speech recognition and preference estimation for each user. In the example above, each terminal device 10 has a speech recognition unit 182 and a preference estimation unit 183, but these may be performed collectively by the server. In this case, the remote meeting server 100 holds the information stored in the memory unit 17 and has the functions of the speech recognition unit 182 and the preference estimation unit 183. The terminal device 10 then functions as a so-called thin client, transmitting information such as user speech and images to the remote meeting server 100 and outputting the information received from the remote meeting server 100. Thus, the remote meeting system 1 may be configured as a so-called centralized system, such as a client-server system where the main processing is performed on the server side.

[0190] Furthermore, in the remote meeting system 1, for example, the terminal device 10 may perform group topic estimation and matching processing. In this case, the terminal device 10 functions as an information processing device that notifies recommended groups. For example, the terminal device 10 holds information stored in the storage unit 120 and has the functions of a data holding unit 132, a group management unit 133, a topic matching unit 134, an output generation unit 135, and a notification unit 136. For example, the audio output unit 13, display 15, processing unit 184, etc. of the terminal device 10 function as the notification unit 136. For example, the terminal device 10 functions as a rich client that performs speech recognition, user preference estimation, group topic estimation, and matching processing. The remote meeting server 100 then collects information from each terminal device 10 and provides each terminal device 10 with the necessary information.

[0191] Furthermore, for example, if each terminal device 10 communicates and shares information to provide remote meeting services among itself in a manner such as so-called P2P (Peer to Peer), the remote meeting system 1 does not necessarily have to include a remote meeting server. Thus, the remote meeting system 1 may have a system configuration in which, for example, the main processing is performed on the user's terminal (client) side and the server only manages information related to the remote meeting, or it may not include a server at all, a so-called autonomous distributed system configuration. As described above, the remote meeting system 1 may have any configuration, such as centralized or autonomous distributed.

[0192] The above-described configuration is merely an example, and the remote meeting system 1 may have any functional division configuration or any device configuration, as long as it can provide the remote meeting services described above.

[0193] [2-2. Others] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0194] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0195] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0196] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0197] [3. Effects of this disclosure] As described above, the information processing device according to this disclosure (for example, a remote meeting server 100 or a terminal device 10 in an embodiment) comprises an estimation unit (a group management unit 133 in an embodiment) and a notification unit (a notification unit 136 in an embodiment). The estimation unit estimates that multiple participants participating in a remote meeting are divided into multiple groups, and that each group can converse within itself, and estimates the topic of conversation within the group from the statements of the participants belonging to the group. The notification unit notifies a participant of the second group if the topic of a second group, which is a group other than the first group to which a participant belongs, satisfies the conditions regarding similarity between the topic of the second group and the preferences of the participant.

[0198] Thus, the information processing device relating to this disclosure can notify participants of appropriate information to those participating in a remote meeting by notifying a participant of groups other than the one to which they belong, based on the similarity between the topics of each group and the preferences of that participant (user).

[0199] Furthermore, the estimation unit estimates the participation status of one participant in the first group. The notification unit changes its notification response regarding the second group according to the participation status of the one participant in the first group estimated by the estimation unit. In this way, the information processing device can provide appropriate information to participants in a remote meeting by changing its notification response according to the participation status of one participant.

[0200] Furthermore, the estimation unit estimates whether a participant's state is a first state, where they are actively participating, or a second state, where they are not in the first state. The notification unit changes the notification method of the second group to the participant depending on whether the participant is in the first or second state. In this way, the information processing device can provide appropriate information to participants in a remote meeting by changing the notification response depending on whether a participant's participation state is a first state, where they are actively participating, or a second state, where they are not in the first state.

[0201] Furthermore, the notification unit changes the timing of the notification of the second group to a participant depending on whether that participant is in the first state or the second state. In this way, the information processing device can notify participants in a remote meeting at an appropriate time by changing the notification timing depending on whether a participant is in the first state, which is an actively participating state, or the second state, which is a state other than the first state.

[0202] Furthermore, the notification unit changes the display method of the second group for a participant depending on whether that participant is in the first state or the second state. In this way, the information processing device can notify participants of remote meetings of appropriate information by changing the display method of the second group depending on whether a participant is in the first state, which is an actively participating state, or in the second state, which is a state other than the first state.

[0203] Furthermore, the notification unit notifies a participant of the second group if that participant is in the second state. In this way, the information processing device can notify a participant of the second group when that participant is in the second state, thereby notifying them when a participant is not actively participating in the first group. Therefore, the information processing device can notify participants of appropriate information when they are taking part in a remote meeting.

[0204] Furthermore, the notification unit notifies a participant of the second group when their participation status transitions from the first state to the second state. In this way, the information processing device can notify a participant of the second group when they transition from the first state to the second state, thereby enabling it to notify a participant when they are no longer actively participating in the first group. Therefore, the information processing device can notify participants in remote meetings at the appropriate time.

[0205] Furthermore, the notification unit superimposes the audio of the participants in the second group onto the audio of the participants in the first group. In this way, the information processing device outputs the audio of the second group along with the audio of the first group in which one participant is participating, allowing one participant to recognize both conversations. Therefore, the information processing device can notify participants taking part in a remote meeting at an appropriate time.

[0206] Furthermore, the notification unit superimposes the voices of the participants in the second group at a second volume, which is lower than the first volume of the first group. In this way, the information processing device outputs the voices of the second group at a lower volume than the voices of the first group in which one participant is participating, allowing the participant to concentrate on the conversation of the group they are in while still being able to hear the voices of the other groups. Therefore, the information processing device can notify participants in a remote meeting at an appropriate time.

[0207] Furthermore, the notification unit displays images of the participants in the second group along with images of the participants in the first group. In this way, the information processing device allows one participant to recognize both groups of participants by displaying images of the participants in the second group along with images of the participants in the first group, in which one participant is a member.

[0208] Furthermore, the notification unit displays the first group and the second group in a way that allows them to be distinguished. In this way, the information processing device allows a participant to recognize which group they belong to and which group they belong to by displaying the first group and the second group in a way that allows them to be distinguished.

[0209] Furthermore, the notification unit displays the images of the participants in the second group in a second area that is smaller than the first area that displays the images of the participants in the first group. In this way, the information processing device displays the first group, in which one participant is a member, larger than the second group, thereby allowing the participant to focus on the group they are in while also being able to recognize the other groups.

[0210] Furthermore, when the notification unit notifies the audio and image of participants in the second group, it notifies them in a manner that allows them to recognize the correspondence between the audio and image of the participants in the second group. In this way, the information processing device can enable a single participant to appropriately recognize the second group by notifying them in a manner that allows them to recognize the correspondence between the audio and image of participants in the second group. Therefore, the information processing device can notify participants taking part in a remote meeting at an appropriate time.

[0211] Furthermore, the notification unit outputs the voice of the participant from the second group from a direction corresponding to the relationship between the display positions of the first group and the second group. In this way, the information processing device can ensure that a participant properly recognizes the second group by outputting the voice of the participant from the second group from a direction corresponding to the relationship between the display positions of the first group and the second group. Therefore, the information processing device can notify participants taking part in a remote meeting at an appropriate time.

[0212] Furthermore, the notification unit notifies a participant of the second group if the group keyword, which is a keyword indicating the topic of the second group, and the participant keyword, which is a keyword indicating the preferences of a participant, satisfy the conditions regarding similarity. In this way, the information processing device can notify participants of remote meetings of groups other than the one they belong to, based on the similarity between the topics of each group and the preferences of the participant (user) using keywords, thereby providing appropriate information to participants participating in remote meetings.

[0213] Furthermore, the notification unit notifies a participant of the second group if the comparison result between the group keyword of the second group and the participant keyword of a participant satisfies the similarity conditions. In this way, the information processing device can provide appropriate information to participants in remote meetings by notifying a participant of a group other than the one to which they belong, when the keyword comparison result satisfies the similarity conditions.

[0214] Furthermore, the notification unit notifies a participant of the second group if the similarity between the group keyword of the second group and the participant keyword of that participant satisfies the similarity conditions. For example, if the keyword similarity is above a predetermined threshold, the information processing device can notify a participant of groups other than the one to which they belong, thereby providing appropriate information to participants in a remote meeting.

[0215] [4. Hardware Configuration] The information processing devices (information equipment) such as the remote meeting server 100 and terminal device 10 according to each embodiment described above are implemented by a computer 1000 having a configuration such as that shown in Figure 14. Figure 14 is a hardware configuration diagram showing an example of a computer 1000 that implements the functions of an information processing device. Hereinafter, the remote meeting server 100 according to the embodiment will be used as an example. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface 1500, and input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050.

[0216] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls various parts. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0217] ROM1300 stores boot programs such as the BIOS (Basic Input Output System) executed by CPU1100 when computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0218] HDD1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU1100 and data used by such programs. Specifically, HDD1400 is a recording medium that records an information processing program related to this disclosure, which is an example of program data 1450.

[0219] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it generates to other devices via the communication interface 1500.

[0220] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0221] For example, when computer 1000 functions as a remote meeting server 100 according to the embodiment, the CPU 1100 of computer 1000 realizes functions such as the control unit 130 by executing an information processing program loaded on RAM 1200. The HDD 1400 stores the information processing program according to this disclosure and data in the storage unit 120. The CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0222] Furthermore, this technology can also be configured as follows. (1) Multiple participants in a remote meeting are divided into multiple groups, each group can converse within its own group, and an estimation unit estimates the topic of conversation within each group based on the statements of the participants belonging to that group. A notification unit notifies the first participant of the second group if the topic of the second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, An information processing device equipped with the following features. (2) The estimation unit, The participation status of the first participant in the first group is estimated, The aforementioned notification unit, Depending on the participation status of the first participant in the first group, as estimated by the estimation unit, the notification response regarding the second group is changed. (1) The information processing device described above. (3) The estimation unit, It is estimated whether the state of the aforementioned participant is a first state, which is a state of active participation, or a second state, which is a state other than the first state. The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the manner in which the second group notifies the first participant is changed. (2) The information processing device described above. (4) The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the timing of the notification of the second group to the first participant is changed. (3) The information processing device described above. (5) The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the display method of the second group to the first participant is changed. The information processing device described in (3) or (4). (6) The aforementioned notification unit, If the first participant is in the second state, the second group is notified to the first participant. An information processing device as described in any one of (3) to (5). (7) The aforementioned notification unit, When the participation status of the first participant transitions from the first state to the second state, the first participant is notified of the second group. An information processing device as described in any one of (3) to (6). (8) The aforementioned notification unit, The audio of the participant from the first group is superimposed with the audio of the participant from the second group. An information processing device described in any one of (1) to (7). (9) The aforementioned notification unit, The voices of the participants in the second group are superimposed at a second volume that is lower than the first volume of the first group. (8) The information processing device described above. (10) The aforementioned notification unit, Display images of the participants in the first group along with images of the participants in the second group. An information processing device described in any one of (1) to (9). (11) The aforementioned notification unit, To display the aforementioned first group and the aforementioned second group in an identifiable manner. (10) The information processing device described above. (12) The aforementioned notification unit, The images of the participants of the second group are displayed in a second region that is smaller than the first region where the images of the participants of the first group are displayed. The information processing device described in (10) or (11). (13) The aforementioned notification unit, When notifying the audio and video of participants in the second group, the correspondence between the audio and video of participants in the second group shall be notified in a manner that allows for recognition. An information processing device described in any one of (1) to (12). (14) The aforementioned notification unit, Notification is given in a manner in which the voice of the participant in the second group is output from a direction corresponding to the relationship between the display positions of the first group and the second group. (13) The information processing device described above. (15) The aforementioned notification unit, If the group keyword, which is a keyword indicating the topic of the second group, and the participant keyword, which is a keyword indicating the preferences of the first participant, satisfy the conditions regarding similarity, the first participant is notified of the second group. An information processing device described in any one of (1) to (14). (16) The aforementioned notification unit, If the comparison result between the group keyword of the second group and the participant keyword of the first participant satisfies the similarity conditions, the first participant is notified of the second group. (15) The information processing device described above. (17) The aforementioned notification unit, If the similarity between the group keyword of the second group and the participant keyword of the first participant satisfies the similarity conditions, the first participant is notified of the second group. The information processing device described in (15) or (16). (18) Multiple participants in a remote meeting are divided into multiple groups, each able to converse within its own group, and the topic of conversation within each group is estimated from the statements made by the participants in that group. If the topic of a second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, the first participant is notified of the second group. An information processing method for determining what to do. [Explanation of Symbols]

[0223] 1. Remote meeting system 100 Remote Meeting Server (Information Processing Device) 110 Communications Department 120 Storage section 121 User Information Storage Unit 122 Group Information Storage Unit 130 Control Unit 131 Acquisition Department 132 Data storage unit 133 Group Management Department 134 Topic Matching Department 135 Output generation unit 136 Notification Department 10. Terminal device (information processing device) 11 Communications Department 12. Voice input section 13 Audio output section 14 Cameras 15. Display (Notification Section) 16 Control section 17 Memory section 18 Control Unit 181 Acquisition Department 182 Voice Recognition Unit 183 Preference Estimation Section 184 Processing Unit

Claims

1. Multiple participants in a remote meeting are divided into multiple groups, each group can converse within its own group, and an estimation unit estimates the topic of conversation within each group based on the statements of the participants belonging to that group. A notification unit notifies the first participant of the second group if the topic of the second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, Equipped with, The aforementioned notification unit, The audio of the participant from the first group is superimposed with the audio of the participant from the second group. Information processing device.

2. Multiple participants in a remote meeting are divided into multiple groups, each group can converse within its own group, and an estimation unit estimates the topic of conversation within each group based on the statements of the participants belonging to that group. A notification unit notifies the first participant of the second group if the topic of the second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, Equipped with, The aforementioned notification unit, Images of the participants in the first group are displayed along with images of the participants in the second group. Information processing device.

3. The estimation unit, The participation status of the first participant in the first group is estimated, The aforementioned notification unit, Depending on the participation status of the first participant in the first group, as estimated by the estimation unit, the notification response regarding the second group is changed. The information processing apparatus according to claim 1.

4. The estimation unit, It is estimated whether the state of the aforementioned participant is a first state, which is a state of active participation, or a second state, which is a state other than the first state. The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the method of notifying the first participant of the second group is changed. The information processing apparatus according to claim 3.

5. The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the timing of the notification of the second group to the first participant is changed. The information processing apparatus according to claim 4.

6. The aforementioned notification unit, Depending on whether the first participant is in the first state or the second state, the display method of the second group to the first participant is changed. The information processing apparatus according to claim 4.

7. The aforementioned notification unit, If the first participant is in the second state, the second group is notified to the first participant. The information processing apparatus according to claim 4.

8. The aforementioned notification unit, When the participation status of the first participant transitions from the first state to the second state, the first participant is notified of the second group. The information processing apparatus according to claim 4.

9. The aforementioned notification unit, The voices of the participants in the second group are superimposed at a second volume that is lower than the first volume of the first group. The information processing apparatus according to claim 1.

10. The aforementioned notification unit, To display the first group and the second group in a way that allows them to be identified. The information processing apparatus according to claim 2.

11. The aforementioned notification unit, The images of the participants of the second group are displayed in a second area that is smaller than the first area where the images of the participants of the first group are displayed. The information processing apparatus according to claim 2.

12. The aforementioned notification unit, When notifying the voice and image of the participants in the second group, the correspondence between the voice and image of the participants in the second group shall be notified in a manner that allows for recognition. The information processing apparatus according to claim 1.

13. The aforementioned notification unit, Notification is given in a manner in which the voice of the participant in the second group is output from a direction corresponding to the relationship between the display positions of the first group and the second group. The information processing apparatus according to claim 12.

14. The aforementioned notification unit, If the group keyword, which is a keyword indicating the topic of the second group, and the participant keyword, which is a keyword indicating the preferences of the first participant, satisfy the conditions regarding similarity, the first participant is notified of the second group. The information processing apparatus according to claim 1.

15. The aforementioned notification unit, If the comparison result between the group keyword of the second group and the participant keyword of the first participant satisfies the similarity conditions, the first participant is notified of the second group. The information processing apparatus according to claim 14.

16. The aforementioned notification unit, If the similarity between the group keyword of the second group and the participant keyword of the first participant satisfies the similarity conditions, the first participant is notified of the second group. The information processing apparatus according to claim 14.

17. Multiple participants in a remote meeting are divided into multiple groups, each able to converse within its own group, and the topic of conversation within each group is estimated from the statements made by the participants in that group. If the topic of a second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, the first participant is notified of the second group. The computer performs the process, The aforementioned computer, The audio of the participant from the first group is superimposed with the audio of the participant from the second group. Information processing methods.

18. Multiple participants in a remote meeting are divided into multiple groups, each able to converse within its own group, and the topic of conversation within each group is estimated from the statements made by the participants in that group. If the topic of a second group (a group other than the first group to which one participant belongs) satisfies the conditions regarding similarity with the preferences of the first participant, the first participant is notified of the second group. The computer performs the process, The aforementioned computer, Images of the participants in the first group are displayed along with images of the participants in the second group. Information processing methods.

Images