Discussion participation activation system, discussion participation activation device, and discussion participation activation method

Abstract

To provide a discussion participation activation system capable of activating group participation, a discussion participation activation device, and a discussion participation activation method.SOLUTION: A discussion activation system includes: a monitoring unit for monitoring the status of participants; a group activity score calculation unit that calculates group activity scores of multiple participants from the information obtained by the monitoring unit; an action determination unit that uses the group activity score as a reward and determines, through reinforcement learning, the action that maximizes the interactivity of the group activity; and an output unit that performs the action determined by the action determination unit.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a discussion participation activation system, a discussion participation activation device, and a discussion participation activation method.

Background Art

[0002] For example, group discussions are held in companies, schools, etc. When participating in a group discussion, people sometimes feel shy. This is a common occurrence among school children. While there are children who do not actively participate in class discussions, it is common to have both children who actively answer questions and share, and those who do not. However, in group discussions, not only individual participation but also group participation is very important. In school, when a teacher feels that the participation rate of the whole group is decreasing, for example, the teacher tries their best to stimulate interaction by speaking to the whole group, eliciting responses, or calling on individuals who are not very active.

[0003] In such a meeting where multiple people participate, a technique for detecting the interaction of participants has been proposed (see, for example, Patent Document 1). In the technique described in Patent Document 1, at least one three-dimensional data stream for two or more participants is captured, time-series skeleton data is extracted from at least one 3D data stream for two or more participants, the time-series skeleton data for each of the two or more participants is classified based on a plurality of body position identifiers, and a participation score for each of the two or more participants is calculated based on the classification of the time-series skeleton data for each of the two or more participants.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technology described in Patent Document 1, even though it was possible to evaluate and provide feedback, it was not possible to maximize group participation.

[0006] The present invention has been made in view of the above problems, and an object thereof is to provide a discussion participation activation system, a discussion participation activation device, and a discussion participation activation method capable of activating group participation.

Means for Solving the Problems

[0007] (1) To achieve the above object, a discussion participation activation system according to an aspect of the present invention includes a monitoring unit that monitors the state of participants, a group activity score calculation unit that calculates a group activity score of a plurality of participants from the information obtained by the monitoring unit, an operation determination unit that uses the group activity score as a reward and determines an operation to maximize the bidirectionality of group activities through reinforcement learning, and an output unit that performs the operation determined by the operation determination unit.

[0008] (2) In the discussion participation activation system according to an aspect of the present invention described in (1) above, the discussion participation activation system further includes a personal activity score calculation unit that acquires the activity status of each participant and calculates a personal activity score obtained by quantifying the activity status, and the group activity score calculation unit may calculate the group activity score based on the personal activity score.

[0009] (3) In the discussion participation activation system according to an aspect of the present invention described in (2) above, the monitoring unit includes a photographing unit and a sound collection unit, the personal activity score calculation unit extracts an image of each participant from the photographed image, extracts a voice signal of each participant from the collected voice signal, and acquires the activity status of each participant using at least one of the extracted images of each participant and the extracted voice signals of each participant.

[0010] (4) In the discussion participation activation system according to one aspect of the present invention as described in (3) above, when the personal activity score calculation unit can acquire a predetermined activity, among a plurality of predetermined activities, a score is assigned to each of the predetermined activities in which the activity is detected, and the personal activity score obtained by totaling the scores of the plurality of predetermined activities and quantifying the activity status may be calculated.

[0011] (5) In the discussion participation activation system according to one aspect of the present invention as described in any one of (2) to (4) above, the output unit acts to make the participants participate through questions and conversations based on the group activity score, acts to talk to the inactive participants based on the personal activity score, and based on the group activity score, if the activity level of the conversation is high, may act to keep silent and continue observing the conversation situation.

[0012] (6) In the discussion participation activation system according to one aspect of the present invention as described in (4) above, at least two of the plurality of predetermined activities are activities in which when the discussion activation device including the output unit or another participant is speaking, the participants are looking at the discussion activation device or the participants are looking at other participants, activities of each of the participants when the discussion activation device asks a question that can be answered with "yes" and "no" to the participants, activities of raising or waving a hand when the participant wants to speak, and activities of nodding when agreeing or not agreeing with the statement of the discussion activation device or another participant.

[0013] (7) In the discussion participation activation system according to one aspect of the present invention as described in any one of (1) to (6) above, the monitoring unit is an environmental sensor, and the environmental sensor is a device separate from the discussion activation device including the output unit, and includes a photographing unit and a microphone array including a plurality of microphones, and may be arranged at a position where the activities of the plurality of participants can be photographed and the voices of the plurality of participants can be collected.

[0014] (8) To achieve the above object, a discussion participation activation device according to an aspect of the present invention includes a monitoring unit that monitors the state of participants, a group activity score calculation unit that calculates a group activity score of a plurality of participants from the information obtained by the monitoring unit, an operation determination unit that uses the group activity score as a reward and determines an operation to maximize the bidirectionality of group activities through reinforcement learning, and an output unit that performs the operation determined by the operation determination unit.

[0015] (9) To achieve the above object, a discussion participation activation method according to an aspect of the present invention includes a monitoring unit that monitors the state of participants, a group activity score calculation unit that calculates a group activity score of a plurality of participants from the information obtained by the monitoring unit, an operation determination unit that uses the group activity score as a reward and determines an operation to maximize the bidirectionality of group activities through reinforcement learning, and an output unit that performs the operation determined by the operation determination unit. [Advantages of the Invention]

[0016] According to the above (1) to (9), group participation can be activated. [Brief Description of the Drawings]

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scales of each member are appropriately changed in order to make each member recognizable in size. In all the drawings for explaining the embodiments, those having the same function are denoted by the same reference numerals, and repeated explanations are omitted. In addition, “based on XX” as used in the present application means “based at least on XX”, and includes cases where it is based on another element in addition to XX. Also, “based on XX” is not limited to the case where XX is directly used, and includes cases where it is based on something obtained by performing an operation or processing on XX. “XX” is an arbitrary element (for example, arbitrary information).

[0019] [Overview] FIG. 1 is a diagram showing a state of group discussion in a school using the discussion activation system of the present embodiment. In the example of FIG. 1, in a school classroom, teacher Te is operating the tablet terminal 5 and projecting an image onto the screen from the projector (image display device 4) (reference sign g11), and children Ch are having a group discussion. In the present embodiment, the robot 2 (discussion activation device) observes the expressions and utterances of the children Ch, calculates the group activity scores of all the participants, and determines an action to maximize the two-way nature of the group activity. Then, the robot 2 performs the determined action.

[0020] In group discussions, instead of teacher Te, robot 2 may display images on image display device 4 or advance the progress of the discussion. In the following embodiments, robot 2 will be described as an example of a discussion activation device, but it is not limited to this. The discussion activation device may be composed of robot 2 and an information processing device or the like. Also, the placement of robot 2 is preferably in front of the children Hu or the like where it can communicate with the children Hu, observe the children Hu, and clearly pick up voices.

[0021] FIG. 2 is a diagram showing a schematic configuration example of the discussion activation system according to the present embodiment. The discussion activation system 1 includes, for example, a robot 2 (discussion activation device), an environment sensor 3, and an image display device 4. The participants in the discussion are, for example, one teacher Te and a plurality of children Ch.

[0022] The children Ch (participants) look at the image of the discussion topic provided by teacher Te on the image display device 4 while communicating with teacher Te. The children Ch make statements and give opinions on the image and the discussion theme.

[0023] Robot 2 is a robot capable of communicating with the children Ch. Robot 2 includes, for example, a main body, a display unit capable of displaying expressions of eyes and mouth, a boom capable of operating the eye part, an audio output unit for outputting an audio signal, a sound collection unit, a camera, a control unit for controlling each functional unit, and the like. During the discussion, robot 2 detects the discussion participation status of the children Ch. Note that robot 2 may obtain the discussion participation status of the children Ch from environment sensor 3, or may use both the information obtained by robot 2 and the information obtained by environment sensor 3. The configuration, operation, processing, etc. of robot 2 will be described later.

[0024] Environment sensor 3 performs, for example, photographing of an area including the face of child Ch and detection of an audio signal. Note that environment sensor 3 may detect the line of sight of child Ch or the like. Environment sensor 3 includes, for example, a stereo camera or an RGBD camera capable of obtaining depth information, and a sound collection unit of a microphone array including a plurality of microphones.

[0025] The image display device 4 displays an image for display. The image display device 4 may be a projector and a screen as shown in FIG. 1. Note that the image display device 4 may include a speaker that outputs an acoustic signal such as a sound effect or BGM (BackGround Music).

[0026] [Example of the external shape of the robot] Next, an example of the external shape of the robot 2 will be described. FIG. 3 is a diagram showing an example of the external shape of the robot according to the present embodiment. In FIG. 3, the front view g101 and the side view g102 are diagrams showing an example of the external shape of the robot 2 according to the embodiment. The robot 2 includes, for example, three display units 111 (eye display unit 111a, eye display unit 111b, mouth display unit 111c). Also, in the example of FIG. 3, the imaging unit 102a is attached to the upper part of the eye display unit 111a, and the imaging unit 102b is attached to the upper part of the eye display unit 111b. The eye display units 111a and 111b correspond to human eyes and present an image or image information corresponding to human eyes. The screen size of the eye display units 111a and 111b is, for example, 3 inches. For example, the voice output unit 112, which is a speaker, is attached in the vicinity of the mouth display unit 111c that displays an image corresponding to the human mouth of the housing 120. The mouth display unit 111c is composed of, for example, a plurality of LEDs (light emitting diodes), each LED can be address-specified, and can be individually driven to turn on and off. The sound collection unit 103 is attached to the housing 120.

[0027] Also, the robot 2 includes a boom 121. The boom 121 is movably attached to the housing 120 via a movable part 131. A horizontal bar 122 is rotatably attached to the boom 121 via a movable part 132. The eye display unit 111a is rotatably attached to the horizontal bar 122 via a movable part 133, and the eye display unit 111b is rotatably attached to the horizontal bar 122 via a movable part 134. Note that the external shape of the robot 2 shown in FIG. 2 is an example and is not limited thereto.

[0028] [Examples of emotional expressions and operation examples of the robot] FIG. 4 is a diagram showing an example of presenting the emotional expression of the robot according to this embodiment in animation. As shown in FIG. 4, the robot 2 presents an emotional expression by changing the animation to be displayed on the display unit 111 (eye display unit 111a, eye display unit 111b, mouth display unit 111c). Examples of the animations for each of the symbols g11 to g17 are "Angry", "Ecstatic", "Disinterested", "Confused", "Blushing", "Sad", and "Sympathetic".

[0029] Note that the emotional expressions and animations shown in FIG. 4 are just examples and are not limited to this. The emotional expressions may be other than those shown in FIG. 4, and the animations for each emotional expression may be different from those in FIG. 4. Also, during each emotional expression, the angle and position of the display unit 111 may be changed as in FIG. 4, the angle of the boom 121 may be changed, or an audio signal may be output together.

[0030] As described with reference to FIGS. 3 and 4, the robot 2 of this embodiment can richly display images of eyes and a mouth on the display unit 111, can richly output an audio signal from the audio output unit, and can communicate with the child Ch by moving the boom 121 and the display unit 111.

[0031] [Configuration Examples of Each Device of the Discussion Activation System] Next, configuration examples of each device of the discussion activation system will be described. FIG. 5 is a diagram showing configuration examples of each device of the discussion activation system according to this embodiment. The robot 2 includes, for example, a monitoring unit 101, a perception unit 106, an operation instruction unit 109 (output unit), an acquisition unit 110, a display unit 111 (output unit), an audio output unit 112 (output unit), a drive unit 113 (output unit), a control unit 114, and a memory unit 115. In FIG. 5, the housing 120, boom 121, horizontal bar 122, movable parts 131, 132, and 134, etc. described with reference to FIG. 3 are omitted for illustration. Also, the robot 2 may not include the imaging unit 102 and the sound collection unit 103. In that case, it may acquire the images and audio signals acquired by the environmental sensor 3. The environmental sensor 3 includes, for example, an imaging unit 301, a sound collection unit 302, and a communication unit 303.

[0032] (Environmental sensor) The imaging unit 301 is, for example, a stereo camera or an RGBD camera capable of obtaining depth information. The imaging unit 301 outputs the captured image to the robot 2 via the communication unit 303.

[0033] The sound collection unit 302 is a microphone array including a plurality of microphones. The sound collection unit 302 outputs the collected audio signal to the robot 2 via the communication unit 303.

[0034] The communication unit 303 outputs the image and the acoustic signal to the robot 2. The timing of transmission is, for example, at regular intervals. The captured image signal includes an image of the faces of the children Ch. The acoustic signal includes the speech of the child Ch.

[0035] (Robot) The monitoring unit 101 includes, for example, an imaging unit 102, a sound collection unit 103, an image processing unit 104, and an audio processing unit 105. The monitoring unit 101 observes the state of the child Ch who is a participant based on at least one of the image captured by the imaging unit 102 and the audio signal collected by the sound collection unit 103.

[0036] During the group discussion, the imaging unit 102 captures the entire group including the faces and upper bodies of each of the children Ch. The imaging unit 102 may be, for example, an RGB (Red, Green, Blue) camera, or may be an RGBD camera that can also acquire depth information D.

[0037] The sound collection unit 103 collects the voices of the children Ch during the discussion. The sound collection unit 103 is a microphone array including a plurality of microphones.

[0038] The image processing unit 104 performs image processing on the images captured by the imaging unit 102 using a well-known method. The image processing unit 104 extracts, for example, a state in which each of the plurality of children Ch participating in the discussion is raising a hand and speaking, or a state in which the mouth is moving continuously for a predetermined time to make a speech.

[0039] The voice processing unit 105 performs voice recognition processing on the voice signals collected by the sound collection unit 103 using a well-known method. The voice processing unit 105 extracts the speaking time and content of each of the plurality of children Ch participating in the discussion.

[0040] The perception unit 106 includes a group activity score calculation unit 107 and an individual activity score calculation unit 108.

[0041] The group activity score calculation unit 107 calculates the group activity scores of a plurality of participants from the information obtained by the monitoring unit 101. Note that the group activity score calculation unit 107 is configured by a network such as, for example, a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). The method for calculating the score will be described later.

[0042] The individual activity score calculation unit 108 obtains the activity status of each participant from the information obtained by the monitoring unit, for example, and calculates an individual activity score by quantifying the activity status. Note that the individual activity score calculation unit 108 is composed of a network such as a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network), for example. The method for calculating the score will be described later.

[0043] The action instruction unit 109 uses the group activity score as a reward and determines, via reinforcement learning, an action that maximizes the bidirectionality of the group activity. The action instruction unit 109 causes the determined action (at least one of the movements of the eyes, mouth, boom 121, horizontal bar 122, etc. in the image, speech, etc.) to be performed.

[0044] The acquisition unit 110 acquires, for example, image data captured from the environment sensor 3 and audio signal data recorded at a predetermined time interval.

[0045] The display unit 111 displays the images of the eyes and mouth generated by the action instruction unit 109.

[0046] The audio output unit 112 outputs the audio signal generated by the action instruction unit 109.

[0047] The drive unit 113 includes, for example, a drive circuit, an actuator, and sensors such as an encoder. The drive unit 113 drives and operates the actuators attached to the movable parts 131 to 134, the boom 121, and the horizontal bar 122 according to the control of the control unit 114.

[0048] The control unit 114 generates a control signal for driving the drive unit 113 using the information output by the action instruction unit 109.

[0049] The storage unit 115 stores programs, threshold values, predetermined values, mathematical formulas used in processing, etc. used for the control of the robot 2.

[0050] Note that the configuration described with reference to FIG. 5 is merely an example and is not limited thereto. For example, there may be a plurality of environmental sensors 3. Also, a part of the functions of the robot 2 may be on the cloud and may be provided via the network NW.

[0051] [Processing of the Discussion Activation System] Next, the processing procedure and processing content of the discussion activation system will be described. FIG. 6 is a diagram showing the processing content of the discussion activation system according to the present embodiment.

[0052] During the discussion, the robot 2 uses the monitoring unit 101 and the perception unit 106 to observe the group activities while observing the reactions and responses of the participants (g101, g103). Note that, in the present embodiment, the monitoring unit 101 and the perception unit 106 constitute a perception system. One of the purposes of the robot 2 is to maximize the group activities. If not, the robot 2 acts using images and sounds especially for individuals with little interaction based on the individual activity score.

[0053] The perception system uses the imaging unit 102 and the sound collection unit 103 to confirm gazing (g114-1,…,g114-N), speaking (g111-1,…,g111-N), raising or waving hands (g112-1,…,g112-N), nodding (g113-1,…,g113-N), and when it corresponds for each child Ch and for each item (“gazing”, “speaking”, “raising or waving hands”, “nodding”), a score of +1 is given to each. Note that the items may be at least two or more of the above four items. Then, the individual activity score calculation unit 108 calculates the total score of the items for each individual (g121-1,…,g121-N).

[0054] In addition, in order to observe the activities of child Ch, the robot 2 asks free-form questions (after which the children talk), or asks questions that can be answered with "yes" or "no" to observe children Ch (the children nod or answer "yes" or "no"). Alternatively, the robot 2 observes the activities of children Ch when the robot 2 is speaking (in this case, the line of sight of children Ch to the robot 2 is expected). Or the robot 2 observes the activities of other children Ch when a specific child is speaking (the line of sight of children Ch to the specific child Ch is expected).

[0055] In this way, the robot 2 monitors the activity scores at the group level and the individual level. In addition, the group activity score calculation unit 107 obtains the group activity score by summing the scores of each of the plurality of children Ch (g131). Note that the group activity score may be obtained by summing the individual activity scores, or may be obtained by weighting and adding the individual activity scores when there are handicaps or the like for children Ch.

[0056] This score is used as a reward (g141) for the robot 2 to select an optimal action. The robot 2 determines the level of group dynamics of the group discussion using the reward (reference sign g151). Then, the robot 2 selects an action based on the estimated situation of the conversation (g153). For example, based on the group activity score, the robot 2 participates in the discussion of children Ch through questions and conversations. Or, based on the individual activity score, the robot 2 changes the direction of its line of sight, especially towards children who are not active. Or, based on the group activity score, if the group activity level seems high, the robot 2 remains silent or the like.

[0057] In this embodiment, in this way, the participation of each of children Ch in the conversation is maximized through the robot 2.

[0058] FIG. 7 is a flowchart of the processing procedure of the discussion activation system according to this embodiment.

[0059] (Step S1) Robot 2 promotes conversation like a teacher and presents topics for discussion to children Ch. For example, in a classroom setting, Robot 2 asks questions of the participants, has them answer, and share their thoughts.

[0060] (Step S2) Using the photographing unit 102 and the sound collection unit 103, Robot 2 observes the activity state “activity reading” of each participant in the discussion.

[0061] (Step S3) The perception unit 106 of Robot 2 observes the activity state through, for example, the following perceptions (a) to (d). (a) Gaze: When the robot or another child is speaking, the children are looking at the robot or the children are looking at another child. (b) Speak: Ask questions that can be answered with “yes” and “no”. (c) Raise or wave a hand: Raise or wave a hand when wanting to answer or share something. (d) Nod: Nod when agreeing or not agreeing. When (a) to (d) are detected, the perception unit 106 is respectively given a score of, for example, +1, and calculates the score of the activities within the group and among the participants during the conversation.

[0062] (Step S4) Taking the measured activities and states of each child Ch as input, the perception unit 106 of Robot 2 calculates the score of the group's activity in the conversation and determines the group's activity level based on the calculated result. Then, the perception unit 106 uses the activity score as a reward value and learns corresponding actions to maximize group participation (interactivity) through reinforcement learning.

[0063] (Step S5) After learning, robot 2 checks the individual activity score to confirm whether there is a child Ch who has not participated sufficiently in the discussion. If there is a child Ch who has not participated sufficiently in the discussion, the value of the action score is low. The reason is that the actions and words of robot 2 must guarantee the value of a well-balanced activity score for each individual child Ch. After learning, robot 2 checks the individual activity score. If there is a child Ch who has not participated sufficiently in the discussion (Step S5; YES), it proceeds to the process of Step S6. After learning, robot 2 checks the individual activity score. If there is no child Ch who has not participated sufficiently in the discussion (Step S5; NO), it proceeds to the process of Step S7.

[0064] (Step S6) The operation instruction unit 109 of robot 2, for example, makes the child participate through questions or conversations, or specially talks to a child Ch who is not very active. After the process, robot 2 returns to the process of Step S2 and repeats the process.

[0065] (Step S7) If the activity level of the conversation is high, for example, the operation instruction unit 109 of robot 2 keeps silent, continues to observe the conversation situation of the children Ch, and returns to the process of Step S2 to repeat the process.

[0066] Note that the processing procedure and processing content using FIG. 7 are just examples and are not limited to this. For example, some processes may be performed simultaneously.

[0067] [Learning of the perception unit] Next, input / output examples during the learning of the perception unit 106 and input / output examples during use will be described. FIG. 8 is a diagram showing input / output examples during the learning of the perception unit and input / output examples during use according to this embodiment. During learning, like the symbol g201, the perception unit 106 receives image data, audio data, and teacher data and outputs an activity score (reward). The image data is, for example, an image of a single child Ch1 that has been image - processed by the image - processing unit 104 from a photographed image. The audio data is, for example, an audio signal of a single child Ch1 that has been audio - processed by the audio - processing unit 105 from a recorded audio signal. Also, the teacher data is correct - answer data and is the individual activity score of that child Ch. The teacher data is obtained, for example, by an administrator observing the interaction between the robot 2 and the child, seeing the child's reaction, and giving a score of +1 to the items with a reaction. The perception unit 106 compares the teacher data with the output difference and repeats learning until the difference is within a predetermined value.

[0068] During use, like the symbol g211, the learned perception unit 106 receives image data and audio data and outputs an activity score (reward).

[0069] Note that for the model of the perception unit 106, image data showing all the children Ch in the group and the audio signals of all the children in the group may be input. In this case, the model of the perception unit 106 may learn both image processing to recognize each child Ch one by one and extract the image data of each child Ch, and learn audio processing to recognize each child Ch one by one and extract the audio data of each child Ch, and then use the network to obtain the activity score of each child Ch.

[0070] Note that in the above - described example, an example where a plurality of children Ch participate in the discussion is described, but it is not limited to this. The participants may be adults, may be a mixture of adults and children, or may be elderly people. Note that the group discussion session may be a study session, a class, a conversation, a presentation, etc.

[0071] Note that in the above - described example, as examples of activities, examples of items such as raising a hand, answering, looking at the robot 2 or other people are described, but it is not limited to this. For example, the perception unit 106 may also use other information such as the facial expression of the child Ch, whether the eyes are closed (dozing) or open (awake) in calculating the score.

[0072] In the above example, the case where the child Ch speaks during the discussion was described, but it is not limited to this. The participant may be, for example, a person who has difficulty speaking. In such a case, even if the participants cannot speak, they can participate in the discussion by typing characters on the keyboard or writing by hand on the tablet. For example, when a participant wants to speak, they can speak by pressing a predetermined button or raising their hand. The robot 2 may seek individual activity scores and group activity scores based on such activities to activate the discussion.

[0073] In addition, when the participant is a person with hearing impairment, the robot 2 may use character information to facilitate the progress of the discussion session and encourage speaking. Even in such a case, the participant can participate in the discussion by typing characters on the keyboard or writing by hand on the tablet.

[0074] As described above, in this embodiment, the robot 2 observes the activities of the child Ch, calculates the group activity scores of all the participants, determines an action that maximizes the bidirectionality of the group activity, and performs the determined action. Thereby, according to this embodiment, it is possible to activate group participation in the discussion session.

[0075] Note that a program for realizing all or part of the functions of the robot 2 in the present invention may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform all or part of the processing performed by the robot 2. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices. Further, the "computer system" is also assumed to include a WWW system having a homepage providing environment (or display environment). Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, etc., and a storage device such as a hard disk built in a computer system. Furthermore, the "computer-readable recording medium" also includes a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and that holds the program for a certain period of time. Alternatively, some or all of these components may be realized by hardware (including a circuit part; circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), SOC (System On Chip), or may be realized by the cooperation of software and hardware.

[0076] In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication wire) like a telephone line. Also, the above program may be for realizing a part of the functions described above. Furthermore, it may be a so-called difference file (difference program) that can realize the functions described above in combination with a program already recorded in the computer system.

[0077] As described above, the embodiments for implementing the present invention have been described using the embodiments. However, the present invention is not limited to such embodiments at all, and various modifications and substitutions can be made without departing from the gist of the present invention.

Explanation of Reference Numerals

[0078] 1... Discussion activation system, 2... Robot, 3... Environmental sensor, 4... Image display device, 101... Monitoring unit, 102... Photographing unit, 103... Sound collection unit, 104... Image processing unit, 105... Voice processing unit, 106... Perception unit, 107... Group activity score calculation unit, 108... Individual activity score calculation unit, 109... Operation instruction unit, 110... Acquisition unit, 111... Display unit, 112... Voice output unit, 113... Driving unit, 114... Control unit, 115... Storage unit, 120... Housing, 121... Boom, 122... Horizontal bar, 131... Movable part, 132... Movable part, 133... Movable part, 134... Movable part, 301... Photographing unit, 302... Sound collection unit, 303... Communication unit

Claims

1. A monitoring unit that monitors the state of participants, A group activity score calculation unit that calculates a group activity score for a plurality of participants from the information obtained by the monitoring unit, An operation determination unit that uses the group activity score as a reward and determines an operation to maximize the bidirectionality of group activities through reinforcement learning, An output unit that performs the operation determined by the operation determination unit, A discussion activation system comprising the above.

2. Further comprising an individual activity score calculation unit that acquires the activity status of each participant and calculates an individual activity score obtained by quantifying the activity status, The group activity score calculation unit calculates the group activity score based on the individual activity score, The discussion activation system according to claim 1.

3. The monitoring unit Comprises a photographing unit and a sound collection unit, The individual activity score calculation unit Extracts the image of each participant from the photographed image and extracts the voice signal of each participant from the collected voice signal, Acquires the activity status of each participant using at least one of the extracted images of each participant and the extracted voice signals of each participant, The discussion activation system according to claim 2.

4. The individual activity score calculation unit When a predetermined activity can be acquired, scores are given to each of the predetermined activities in which the activity is detected among the plurality of predetermined activities, and the activity status is quantified by summing the scores of the plurality of predetermined activities to calculate an individual activity score, The discussion activation system according to claim 3.

5. The output unit Acts to involve the participants through questions and conversations based on the group activity score, Acts to talk to the inactive participants based on the individual activity score, Based on the group activity score, if the activity level of the conversation is high, acts to maintain silence and continue observing the conversation situation, The discussion activation system according to claim 2 or claim 3.

6. The plurality of predetermined activities are An activity in which the participants are looking at the discussion activation device or the participants are looking at other participants when the discussion activation device including the output unit or another participant is speaking, The activities of each of the participants when the discussion activation device asks the participants questions that can be answered with "yes" and "no", The activity of raising or waving a hand when the participant wants to speak, and the activity of nodding when agreeing or not agreeing with the speech of the discussion activation device or other said participants, wherein at least two of them are, The discussion activation system according to claim 4.

7. The monitoring unit is an environmental sensor, The environmental sensor is, a device separate from the discussion activation device including the output unit, comprising a photographing unit and a microphone array including a plurality of microphones, configured to be able to photograph the activities of the plurality of participants and arranged at a position where the voices of the plurality of participants can be picked up, The discussion activation system according to claim 1 or claim 2.

8. A monitoring unit that monitors the state of participants, A group activity score calculation unit that calculates a group activity score of a plurality of participants from the information obtained by the monitoring unit, An operation determination unit that uses the group activity score as a reward and determines an operation to maximize the two-way nature of group activities through reinforcement learning, An output unit that performs the operation determined by the operation determination unit, A discussion activation device comprising.

9. The monitoring unit monitors the state of participants, The group activity score calculation unit calculates a group activity score of a plurality of participants from the information obtained by the monitoring unit, The operation determination unit uses the group activity score as a reward and determines an operation to maximize the two-way nature of group activities through reinforcement learning, The output unit performs the operation determined by the operation determination unit, A discussion activation method.

Images