Support system, control program, and control method

Abstract

To provide a cheering system that can give people a sense of unity and satisfaction. [Solution] The cheering system 10 includes an audience robot 16 that performs cheering actions together with humans, and cheering tools (user penlights 22) used by humans. The cheering system 10 also includes motion detection means for detecting the movement of the cheering tools, emotion state estimation means for estimating the emotional state of humans based on the movement of the cheering tools, and robot control means for causing the audience robot to perform cheering actions that match the emotional state of humans.

Description

Technical Field

[0001] The present invention relates to a support system, a control program, and a control method, and particularly, for example, to a support system, a control program, and a control method including a robot that performs a support operation together with a person participating in an event.

Background Art

[0002] An example of a conventional support system of this type is disclosed in Patent Document 1. In the technology of Patent Document 1, in a place where a plurality of performers transmit information to a plurality of audiences, transmission information signal detection means for detecting the transmission information of the performers, and from the transmission information signal data of the transmission information of the performers detected by the transmission information signal detection means, feature extraction means for extracting the features of the transmission information signal data and converting it into transmission information signal feature data, and a plurality of information transmission means and transmission means for transmitting the transmission information signal feature data from the performer to the audience and transmitting the transmitted information. At the same time, one of the audiences holds a support item for supporting one of the performers. Then, the support item includes physical operation means that physically operates based on the transmission information of the performer, selects one of the plurality of information transmission means and transmission means, and selectively receives one of the transmission information signal feature data from the information transmission means and transmission means. And control means for controlling the physical operation of the physical operation means based on the transmission information signal feature data received by the selective reception means, thereby reinforcing the information transmission between one of the performers and one of the audiences.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0004] In live concerts and sporting events, the experience of performing the same cheering actions as a group (multiple people) and synchronizing movements and emotions can provide people with a strong sense of unity and satisfaction. If such experiences could be shared with robots or virtual characters (CG agents) instead of other people, it would be useful in improving people's daily well-being. However, such a cheering system has not yet been proposed.

[0005] Therefore, the primary objective of this invention is to provide a novel support system, control program, and control method.

[0006] Another object of this invention is to provide a cheering system, control program, and control method that can give humans a sense of unity and satisfaction. [Means for solving the problem]

[0007] The first invention is a cheering system comprising a robot that performs cheering actions together with a human, the system comprising: a cheering tool used by the human; a motion detection means for detecting the movement of the cheering tool; an emotion state estimation means for estimating the human's emotional state based on the movement of the cheering tool detected by the motion detection means; and a robot control means for causing the robot to perform cheering actions that match the human's emotional state estimated by the emotion state estimation means.

[0008] According to the first invention, since the robot performs cheering actions in accordance with the human's emotional state together with the human, it can give the human a strong sense of unity and satisfaction, and can improve the human's sense of well-being.

[0009] The second invention is subordinate to the first invention and further comprises a display device for displaying a virtual character that performs cheering actions together with a human, and a display control means for causing the virtual character to perform cheering actions that match the human's emotional state estimated by the emotional state estimation means.

[0010] According to the second invention, in addition to the robot, a virtual character performs cheering actions in accordance with the human's emotional state, together with the human, which can give the human a stronger sense of unity and satisfaction, and further enhance their sense of well-being.

[0011] The third invention is dependent on the first or second invention and further comprises a second cheering tool used by a robot, and a tool control means for causing the second cheering tool to perform actions that match the human emotional state estimated by the emotional state estimation means.

[0012] According to the third invention, in addition to the robot, the second cheering device performs cheering actions that match the human emotional state, which can give humans a stronger sense of unity and satisfaction, and further enhance their sense of well-being.

[0013] The fourth invention is dependent on the first or second invention, wherein the movement of the cheering device detected by the motion detection means includes the acceleration and displacement of the cheering device.

[0014] According to the fourth invention, it is possible to appropriately estimate a person's emotional state.

[0015] The fifth invention is subordinate to the first or second invention and further comprises a second robot that is the object of human support.

[0016] The sixth invention is a cheering system comprising a cheering agent that performs cheering actions together with a human, the system comprising: a cheering tool used by the human; a motion detection means for detecting the movement of the cheering tool; an emotional state estimation means for estimating the emotional state of the human based on the movement of the cheering tool detected by the motion detection means; and an agent control means for causing the cheering agent to perform cheering actions that match the emotional state of the human estimated by the emotional state estimation means.

[0017] According to the sixth invention, since support agents such as robots and virtual characters perform support actions in accordance with the human's emotional state together with the human, it is possible to give the human a strong sense of unity and satisfaction, thereby improving the human's sense of happiness.

[0018] The seventh invention is a control program executed in a control device of a cheering system equipped with a cheering agent that performs cheering actions together with a human, wherein the processor of the control device functions as a motion detection means for detecting the movement of cheering tools used by a human, an emotion state estimation means for estimating the emotional state of a human based on the movement of the cheering tools detected by the motion detection means, and an agent control means for causing the cheering agent to perform cheering actions that match the emotional state of the human estimated by the emotion state estimation means.

[0019] The eighth invention is a control method for a control device of a cheering system equipped with a cheering agent that performs cheering actions together with a human, wherein the processor of the control device detects the movement of cheering tools used by the human, estimates the human's emotional state based on the detected movement of the cheering tools, and causes the cheering agent to perform cheering actions that match the estimated human's emotional state.

[0020] The seventh and eighth inventions also produce the same effects as the sixth invention. [Effects of the Invention]

[0021] This invention can provide humans with a strong sense of unity and satisfaction, thereby improving their sense of well-being.

[0022] The aforementioned objectives, other objectives, features, and advantages of this invention will become even clearer from the following detailed description of embodiments with reference to the drawings. [Brief explanation of the drawing]

[0023] [Figure 1] This diagram schematically shows an event venue where a cheering system according to one embodiment of this invention is installed. [Figure 2] It is a block diagram showing an example of the configuration of the support system. [Figure 3] It is a block diagram showing an example of the electrical configuration of the control device included in the support system. [Figure 4] It is a diagram showing the appearance of the support robot included in the support system. [Figure 5] It is a block diagram showing an example of the electrical configuration of the support robot. [Figure 6] It is a diagram schematically showing the state of the spectator robot and the virtual character that change the support action according to the emotional state of the user. [Figure 7] It is an illustrative diagram showing an example of the memory map of the RAM of the control device. [Figure 8] It is a flowchart showing an example of the support control process executed by the CPU of the control device.

Best Mode for Carrying Out the Invention

[0024] Referring to FIGS. 1 and 2, a support system 10 which is an embodiment of this invention includes a plurality of spectator robots 16 and the like that execute a support action together with a user (human) participating in an event. As will be described in detail later, in the support system 10, a plurality of spectator robots 16 and the like synchronize their movements and emotions with the user to execute a support action, so that even when the user participates in the event alone (one person), the user is given a sense of unity and satisfaction equivalent to the case where a group (a plurality of humans) performs the same support action. Hereinafter, the case where the support system 10 is applied to the live concert of the idol robot 24 will be exemplified and described. <00​​As shown in Figures 1 and 2, the cheering system 10 includes a control device 12 that comprehensively controls the entire cheering system 10. The control device 12 is a general-purpose personal computer or workstation, or other computer (information processing device). Multiple spectator robots 16, display devices 18, multiple robot penlights 20, user penlights 22, and idle robots 24 are connected to this control device 12 via a network 14 such as a LAN and the Internet, or via an input / output interface 36 (see Figure 3). These may be connected by wire or wirelessly.

[0026] Figure 3 is a block diagram showing an example of the electrical configuration of the control device 12. As shown in Figure 3, the control device 12 includes a CPU 30. The CPU 30 is connected to the memory 32, the communication interface 34, and the input / output interface 36 via an internal bus.

[0027] The CPU 30 is a processor that oversees the overall control of the control device 12 and, by extension, the support system 10. The CPU 30 functions as agent control means such as robot control means and display control means according to this invention, as well as motion detection means, emotion state estimation means and tool control means.

[0028] Memory 32 includes RAM, ROM, and HDD, etc. The CPU 30 can control the spectator robot 16, display device 18, robot penlight 20, and idle robot 24 by executing various programs stored in memory 32, as described later. Each program is pre-stored in ROM or HDD and is loaded into RAM and executed as needed.

[0029] The communication interface 34 is an interface for sending and receiving control signals and data to and from external devices such as the spectator robot 16, the robot penlight 20, and the idle robot 24 via the network 14 under the control of the CPU 30.

[0030] The input / output interface 36 is connected to input devices such as a keyboard and computer mouse, as well as output devices such as a monitor, as appropriate (though not shown in the diagram). Furthermore, the input / output interface 36 is connected to a display device 18 for displaying the virtual character 26 (CG agent), which will be described later, and a speaker 28 for outputting music from a live concert. The input / output interface 36 outputs operation data (operation information) received from the input devices to the CPU 30. The input / output interface 36 also outputs image data and sound data generated by the CPU 30 to the display device 18 and speaker 28, etc., to display the virtual character 26 on the display device 18 and output music from the speaker 28. Note that the configuration of the control device 12 shown in Figure 3 is just an example and is not limited to this configuration.

[0031] Referring to Figure 4 along with Figure 1, the spectator robot 16 is a robot that performs cheering actions together with the user at a location near the user, and is an example of a cheering agent. In this embodiment, RoboBee®, a humanoid robot manufactured and sold by Vstone Corporation, is used. However, any robot with a different appearance and structure can be used as the spectator robot 16, as long as it can perform cheering actions such as waving a penlight. For example, Sota® and its puppet version (puppet robot) manufactured by Vstone Corporation can be suitably used.

[0032] In short, the spectator robot 16 includes a trolley 50, and multiple wheels 52 are provided on the underside of the trolley 50. Each of the multiple wheels 52 is independently driven by a wheel motor 54 (see Figure 5), allowing the trolley 50 (and thus the spectator robot 16) to move in any direction, forward, backward, left, or right. A body 56 is mounted upright on top of the trolley 50. A distance sensor 58, such as an infrared distance sensor or an ultrasonic distance sensor, is provided on the upper center of the front of the body 56. This distance sensor 58 measures the distance to an object in front of the spectator robot 16.

[0033] Upper arms 62R and 62L are provided at the upper ends of both sides of the torso 56 via shoulder joints 60R and 60L. Each of the shoulder joints 60R and 60L has three orthogonal degrees of freedom, allowing the angles of the upper arms 62R and 62L to be controlled around each of these three orthogonal axes. Forearms 66R and 66L are provided at the ends of the upper arms 62R and 62L via elbow joints 64R and 64L. Each of the elbow joints 64R and 64L has one degree of freedom, allowing the angles of the forearms 66R and 66L to be controlled around this axis. Furthermore, right hands 68R and left hands 68L, which have the functions of human fingers and palms, are provided at the ends of the forearms 66R and 66L.

[0034] A head 72 is provided at the upper end of the torso 56 via a neck joint 70. The neck joint 70 has three degrees of freedom, and the angle of the head 72 can be controlled around each of these three axes. A speaker 74 is provided at the bottom of the head 72, and microphones 76R and 76L are provided on both sides of the head 72. In addition, eyeball sections 78R and 78L are provided on both sides of the front of the head 72, and eye cameras 80R and 80L are provided on these eyeball sections 78R and 78L, respectively.

[0035] Figure 5 is a block diagram showing the electrical configuration of the spectator robot 16. As shown in Figure 5, the spectator robot 16 includes a CPU 82. The CPU 82 is the processor that oversees the overall control of the spectator robot 16 and controls its operation based on commands (operation instructions) from the CPU 30 of the control unit 12. The CPU 82 is connected to a memory 84, a motor control board 86, a sensor input / output board 88, and an audio input / output board 90 via a bus.

[0036] Memory 84 includes RAM, ROM, and HDD. RAM is used as work memory and buffer memory for CPU 82. Control programs (robot control programs) for controlling the movements (executing tasks) of the spectator robot 16 are stored in ROM and HDD. For example, the robot control program includes an arm swinging program that controls the shoulder joints 60R, 60L and elbow joints 64R, 64L, etc., to swing the arm portion (and consequently the robot penlight 20 described later), including the upper arm 62R, 62L, forearm 66R, 66L, and hand 68R, 68L. The robot control program also includes detection programs for detecting the output of each sensor (sensor information), and communication programs for sending and receiving necessary data and commands with an external computer such as the control device 12.

[0037] The motor control board 86, for example, is composed of a DSP and controls the drive of each axis motor, such as those for the shoulder joints 60R and 60L, elbow joints 64R and 64L, and neck joint 70. Specifically, the motor control board 86 receives control data from the CPU 82 and controls the rotation angles of a total of four motors (collectively referred to as "right arm motor 92" in Figure 5): three motors that control the angles of the three orthogonal axes of the shoulder joint 60R and one motor that controls the angle of the elbow joint 64R. Similarly, the motor control board 86 receives control data from the CPU 82 and controls the rotation angles of a total of four motors (collectively referred to as "left arm motor 94" in Figure 5): three motors that control the angles of the three orthogonal axes of the shoulder joint 60L and one motor that controls the angle of the elbow joint 64L. In addition, the motor control board 86 receives control data from the CPU 82 and controls the rotation angles of three motors (collectively referred to as "head motor 96" in Figure 5) that control the angles of the three orthogonal axes of the neck joint 70. Furthermore, the motor control board 86 receives control data from the CPU 82 and controls the rotation angle of the two motors that drive the wheels 52 (collectively referred to as "wheel motors 98" in Figure 5).

[0038] The sensor input / output board 88, like the motor control board 86, is composed of a DSP and receives signals from each sensor and provides them to the CPU 82. Specifically, distance data from the distance sensor 58 (for example, reflection time data) is input to the CPU 82 through this sensor input / output board 88. Similarly, video signals from the eye cameras 80R and 80L are also input to the CPU 82. The audio input / output board 90 is also composed of a DSP and outputs voice or speech from the speaker 74 according to the speech synthesis data provided by the CPU 82. Audio input from microphones 76R and 76L is also provided to the CPU 82 via the audio input / output board 90.

[0039] Furthermore, a communication unit 100 is connected to the CPU 82 via a bus. The communication unit 100 transmits data received from the CPU 82 to an external computer such as the control unit 12, and also provides data and commands received from the external computer to the CPU 82. Note that the configuration of the spectator robot 16 shown in Figures 4 and 5 is just one example and is not limited to this configuration.

[0040] Returning to Figures 1 and 2, the display device 18 is installed in a position easily visible to the user, such as behind the idle robot 24. The display device 18 is preferably one with a large display surface, and projectors and flat panel displays are suitably used. Alternatively, the display device 18 may be a transparent head-mounted display. In this embodiment, a projector that projects images (videos) onto a display surface such as a wall or screen is used as the display device 18. The projector includes a projection device, a control unit, memory, and a communication interface. A commercially available projector can be used as appropriate, so a detailed explanation is omitted.

[0041] The display surface of the display device 18 displays a virtual character 26, which is another example of a cheering agent that performs cheering actions together with the user. The virtual character 26 is a virtual cheering agent (CG agent) rendered using computer graphics, and in this embodiment, it is an image or video of multiple penlights or multiple spectators holding penlights. The image data or video data of this virtual character 26 is generated by the CPU 30 of the control device 12. The CPU 30 transmits the generated image data of the virtual character 26 to the display device 18, causing the image of the virtual character 26 to be displayed on the display device 18. In other words, the cheering actions of the virtual character 26 (such as the movement and light emission patterns of the penlights) are controlled based on the operation instructions from the CPU 30.

[0042] The robot penlight 20 is an example of a second cheering tool used by a robot and is attached to one or both of the hands 68R, 68L of the spectator robot 16. The robot penlight 20 can emit light in multiple patterns with different colors, brightness, and flashing intervals. The robot penlight 20 is communicatively connected to the control device 12 via a microcontroller board such as an Arduino® (not shown). The operation (lighting pattern) of the robot penlight 20 is controlled based on operation instructions from the CPU 30 of the control device 12.

[0043] The user penlight 22 is an example of a cheering tool used by a user, and is held in one or both of the user's hands. The user penlight 22 can emit light in multiple lighting patterns with different colors, brightness, and flashing intervals based on user operations. The user penlight 22 also incorporates motion sensors such as an accelerometer or gyroscope to detect the movement of the user penlight 22, as well as a communication unit. Sensor information detected by the motion sensor is transmitted to the control device 12 via the communication unit. For example, the CPU 30 of the control device 12 calculates the acceleration and amount of movement (amplitude) when the user penlight 22 is swung, based on the sensor information detected by the motion sensor.

[0044] The idol robot 24 is an example of a second robot that serves as the object of the user's support (the one the user cheers for), and performs live at a live concert. The idol robot 24 may be a humanoid robot similar to the audience robot 16. In this embodiment, the operation of the idol robot 24 is controlled based on operation instructions from the CPU 30 of the control device 12, similar to the audience robot 16.

[0045] In this type of cheering system 10, the audience robot 16 and the virtual character 26 displayed on the display device 18 cheer for the idol robot 24 together with the user. At this time, the audience robot 16, the robot penlight 20, and the virtual character 26 perform cheering actions that synchronize with the user's movements and emotions. To put it simply, in this cheering system 10, the movement (specifically acceleration and displacement) of the user penlight 22 when the user waves the user penlight 22 to cheer for the idol robot 24 is detected at predetermined time intervals, for example, every 3 seconds or every 5 seconds. It is assumed that as the user becomes more immersed in the live concert (i.e., their emotions intensify), they will wave the user penlight 22 more vigorously (i.e., larger and faster), so the control device 12 estimates the user's emotional state based on the degree of vigor in which the user penlight 22 is waved. In this embodiment, the control device 12 performs emotion estimation using pre-set threshold values ​​for acceleration and movement, and causes the audience robot 16, robot penlight 20, and virtual character 26 to perform cheering actions that match the estimated emotional state.

[0046] The operation of the support system 10 will be explained in detail below with reference to Figure 6. Note that a1 and b1 shown below are threshold values ​​for movement, and a2 and b2 are threshold values ​​for acceleration. Furthermore, a1 is assumed to be smaller than a2, and b1 is assumed to be smaller than b2.

[0047] As shown in Figure 6(A), if the user is not waving the user penlight 22 very much, for example, if the amount of movement of the user penlight 22 is less than threshold a1 and the acceleration of the user penlight 22 is less than threshold b1, it is estimated that the user's emotional state is low. When the user's emotional state is low, the audience robots 16, robot penlights 20, and virtual character 26 offer weak support. For example, the audience robots 16 may wave the robot penlights 20 slowly and in small amounts, or the number of audience robots 16 waving the robot penlights 20 may be reduced. The robot penlights 20 may also emit light in a subdued color or blink slowly. Furthermore, the virtual character 26 may wave its penlight slowly and in small amounts, or the number of penlights being waved may be reduced.

[0048] Furthermore, as shown in Figure 6(B), if the user is waving the user penlight 22 moderately, for example, if the amount of movement of the user penlight 22 is between threshold a1 and a2 and the acceleration of the user penlight 22 is between threshold b1 and b2, then the user's emotional state is estimated to be moderate. When the user's emotional state is moderate, the audience robots 16, the robot penlights 20, and the virtual character 26 provide moderate support. For example, the audience robots 16 may wave the robot penlights 20 a little more vigorously and quickly, or the number of audience robots 16 waving the robot penlights 20 may increase. The robot penlights 20 may also emit light in a slightly brighter color or blink faster. In addition, the virtual character 26 may wave its penlight a little more vigorously and quickly, or the number of penlights being waved may increase.

[0049] Furthermore, as shown in Figure 6(C), if the user is vigorously waving the user penlight 22, for example, if the amount of movement of the user penlight 22 is greater than or equal to threshold a2 and the acceleration of the user penlight 22 is greater than or equal to threshold b2, it is estimated that the user's emotional state is high. When the user's emotional state is high, the audience robots 16, robot penlights 20, and virtual character 26 cheer with great enthusiasm. For example, all audience robots 16 wave their robot penlights 20 vigorously and quickly. The robot penlights 20 also emit bright colors and flash rapidly. In addition, all the penlights of the virtual character 26 are waved vigorously and quickly.

[0050] In this way, the cheering system 10 increases the intensity of the cheering actions of the audience robots 16, etc., as the user's emotions intensify, or increases the number of audience robots 16, etc. that move more dramatically, thereby increasing the excitement of the cheering actions of the audience robots 16, etc. In other words, the audience robots 16, etc., do not simply imitate the user's cheering actions, but perform cheering actions that match the user's emotional state. As a result, even when a user is cheering for the idol robot 24 alone, it is possible to give the user the same sense of unity and satisfaction as when many people are performing the same cheering action.

[0051] Figure 7 shows an example of the memory map 200 of the RAM built into the control device 12. As shown in Figure 7, the RAM of the control device 12 includes a program storage area 202 and a data storage area 204. The program storage area 202 stores the cheer control program executed by the control device 12. The cheer control program includes a main processing program 202a, a communication program 202b, a user penlight information acquisition program 202c, an emotion estimation program 202d, an audience robot control program 202e, a display control program 202f, a robot penlight control program 202g, and an idle robot control program 202h, etc.

[0052] The main processing program 202a is a program for executing the main routine of the cheer control processing of the control device 12 in this embodiment. The communication program 202b is a program for communicating (sending and receiving data, etc.) with external devices such as the spectator robot 16. The user penlight information acquisition program 202c is a program for calculating the acceleration and movement amount when the user penlight 22 is waved, based on sensor information transmitted from the motion sensor of the user penlight 22. The emotion estimation program 202d is a program for estimating the user's emotional state by comparing the acceleration and movement amount of the user penlight 22 with thresholds.

[0053] The spectator robot control program 202e is a program for controlling the cheering actions of the spectator robot 16 to match the user's emotional state, and includes a program for selecting and sending commands (action instructions) that indicate the intensity (speed and size) of the arm movements of the spectator robot 16. The display control program 202f is a program for generating and sending image data or video data necessary to display the virtual character 26, which performs cheering actions that match the user's emotional state, on the display device 18.

[0054] The robot penlight control program 202g is a program for controlling the cheering behavior of the robot penlight 20 to match the user's emotional state, and includes a program for selecting and sending commands that indicate the light emission pattern of the robot penlight 20. The idle robot control program 202h is a program for causing the idle robot 24 to perform a live performance.

[0055] Although not shown in the diagram, the program storage area 202 also stores other application programs, etc., in addition to the support control program of this invention.

[0056] Meanwhile, the data storage area 204 stores user penlight data 204a, emotion estimation data 204b, robot motion data 204c, image generation data 204d, and robot penlight data 204e, among others.

[0057] User penlight data 204a consists of sensor information transmitted from the motion sensor of the user penlight 22, and data on the acceleration and movement of the user penlight 22 calculated based on this information. Emotion estimation data 204b consists of threshold data for acceleration and movement to estimate the user's emotional state, as well as data indicating the estimated emotional state of the user (whether it is low, medium, or high).

[0058] Robot motion data 204c is data about commands that indicate robot motion patterns prepared in response to the user's emotional state. Image generation data 204d is data such as polygon data and texture data used to generate image data or video data of the virtual character 26. Robot penlight data 204e is data about commands that indicate the emission patterns of the robot penlight 20 prepared in response to the user's emotional state.

[0059] Although not shown in the diagram, the data storage area 204 stores other data necessary for the control device 12 to perform support control processing, and also contains timers (counters) and flags necessary for performing support control processing.

[0060] Figure 8 is a flowchart showing an example of the cheer control processing performed by the CPU 30 of the control device 12. As shown in Figure 8, when the cheer control processing is started, the CPU 30 first starts the audience robots 16, the display device 18, and the robot penlights 20 in step S1. That is, it starts the audience robot control program 202e, performs an access check with each audience robot 16, and sends an activation signal to the audience robots 16 that pass the access check. Similarly, it also starts the display control program 202f and the robot penlight control program 202g. In the next step 3, the live concert of the idle robot 24 is started. Here, music is played from the speaker 28, and the idle robot control program 202h is started to cause the idle robot 24 to start a live performance.

[0061] In the following step S5, motion information is acquired from the user penlight 22. Specifically, sensor information is acquired from the user penlight 22 to calculate the acceleration and displacement when the user penlight 22 is shaken. In step S7, the user's emotional state is estimated based on the motion information (acceleration and displacement) acquired from the user penlight 22.

[0062] In the subsequent step S9, the system sends action instructions to the spectator robots 16, the display device 18, and the robot penlight 20, corresponding to the estimated emotional state of the user. In response, each spectator robot 16 performs a cheering action appropriate to the user's emotional state. When a series of cheering actions is completed, the system sends an action completion signal to the CPU 30 of the control device 12. Similarly, the display device 18 displays a virtual character 26 on its display surface that performs a cheering action appropriate to the user's emotional state. The robot penlight 20 also emits light in a pattern appropriate to the user's emotional state.

[0063] In the following step S11, it is determined whether a signal indicating the end of operation has been received from the spectator robot 16. If the answer in step S11 is "NO," that is, if a signal indicating the end of operation has not been received from the spectator robot 16, the system waits until a signal indicating the end of operation is received. On the other hand, if the answer in step S11 is "YES," that is, if a signal indicating the end of operation has been received from the spectator robot 16, the system proceeds to step S13. In step S13, it is determined whether there is an end instruction. For example, if the end button is operated on the control device 12 or if the series of live performances has ended, it is determined that there is an end instruction. If the answer in step S13 is "NO," that is, if there is no end instruction, the system returns to step S5. On the other hand, if the answer in step S13 is "YES," that is, if there is an end instruction, the system terminates this cheering control process.

[0064] Note that the processing steps in the flowchart shown in Figure 8 are merely examples, and the order of processing steps can be changed if similar results can be obtained.

[0065] As described above, according to this embodiment, the audience robot 16, robot penlight 20, and virtual character 26 perform cheering actions in accordance with the user's emotional state together with the human, thereby giving the human a strong sense of unity and satisfaction, and improving the human's sense of well-being.

[0066] In the above-described embodiment, a penlight was used as an example of a cheering tool used by the user and the robot, but it is not limited to this. Cheering tools may also include fans, towels and wristbands used at live concerts, or cheering bats, balloons and megaphones used at baseball games, etc.

[0067] Furthermore, in the above-described embodiment, both audience robots and virtual characters (CG agents) were used as cheering agents, but the cheering system may include only one of them. The cheering agent in this invention refers to a proxy entity capable of performing cheering actions on behalf of other audience members at an event in which a user participates, and includes robots existing in real space and characters existing in virtual space.

[0068] Furthermore, although the above embodiment includes multiple spectator robots, only one spectator robot is sufficient. However, to more reliably provide users with a strong sense of unity and satisfaction, it is preferable to include multiple spectator robots (for example, three or more). Also, while the event to which this cheering system is applied is basically attended by one person (i.e., enjoyed individually), it is also possible to participate with other people. In this case, the movement (acceleration and displacement) of the cheering tools used by the user should be the average of that of multiple users.

[0069] Furthermore, while the above-described embodiment uses an idol robot as the target of the user's cheering, it is not limited to this. For example, the target of the cheering could be a real-life artist or a sport such as baseball or soccer. Also, the target of the cheering does not necessarily have to be in the same space as the user; as long as the environment allows for the placement of a cheering agent such as a robot or virtual character (display device) near the user, the user can participate in the event remotely (online).

[0070] Furthermore, in the above-described embodiment, the idol robot that is the target of the user's cheering actions performs a live performance according to a predetermined control program, but the idol robot's movements may also be changed in accordance with the user's emotional state. For example, as the user's emotional state intensifies, the idol robot's gestures and movements may become more intense. [Explanation of symbols]

[0071] 10... Support System 12 ... control device 16... Audience robots (robots, support agents) 18...display device 20…Robot penlight (second cheering tool) 22... User-operated penlight (cheering prop) 24... Idol Robot (Second Robot) 26…Virtual Character (Support Agent) 30 ...CPU (Processor) of the control unit

Claims

1. A support system equipped with robots that perform support actions together with humans, The cheering tools used by the aforementioned humans, Motion detection means for detecting the movement of the aforementioned cheering equipment, An emotional state estimation means for estimating the emotional state of the person based on the movement of the cheering tool detected by the motion detection means, and A cheering system comprising robot control means for causing the robot to perform cheering actions that match the human emotional state estimated by the emotional state estimation means.

2. A display device that displays a virtual character that performs cheering actions together with the aforementioned human, and The cheering system according to claim 1, further comprising a display control means for causing the virtual character to perform cheering actions that match the human emotional state estimated by the emotional state estimation means.

3. A second cheering tool used by the robot, and The cheering system according to claim 1 or 2, further comprising a tool control means for causing the second cheering tool to perform an action that matches the emotional state of the person estimated by the emotional state estimation means.

4. The cheering system according to claim 1 or 2, wherein the movement of the cheering tool detected by the motion detection means includes the acceleration and displacement of the cheering tool.

5. The cheering system according to claim 1 or 2, further comprising a second robot which is the target of the human cheering.

6. A support system equipped with support agents that perform support actions together with humans, The cheering tools used by the aforementioned humans, Motion detection means for detecting the movement of the aforementioned cheering equipment, An emotional state estimation means for estimating the emotional state of the person based on the movement of the cheering tool detected by the motion detection means, and A cheering system comprising agent control means for causing the cheering agent to perform cheering actions that match the human emotional state estimated by the emotional state estimation means.

7. A control program executed in the control device of a cheering system equipped with cheering agents that perform cheering actions together with humans, The processor of the control device is Motion detection means for detecting the movement of cheering equipment used by the aforementioned human, An emotional state estimation means for estimating the emotional state of the person based on the movement of the cheering tool detected by the motion detection means, and A control program that functions as an agent control means, causing the support agent to perform a support action that matches the human emotional state estimated by the emotional state estimation means.

8. A control method for a control device of a cheering system equipped with a cheering agent that performs cheering actions together with a human, The processor of the control device is The movement of the cheering equipment used by the aforementioned human is detected, Based on the detected movement of the cheering device, the emotional state of the person is estimated, and A control method for causing the support agent to perform a support action that matches the estimated emotional state of the person.

Images