Method for conducting a performance event and relay device used in said method for conducting a performance event

Abstract

To provide a performance event execution method which enables a player to perceive in real time reactions of a remote viewer located in a remote place from an event venue and enables the remote viewer to perceive in real time reactions of other remote viewers.SOLUTION: Each of remote viewers possesses a terminal device for remote viewer. An avatar device generates a live video signal and a live sound signal on the condition that a plurality of avatar devices associated with the respective remote viewers are installed in an event venue. The terminal device for remote viewer displays a live video on the basis of the live video signal and emits live sound on the basis of the live sound signal, whereas the terminal devices for remote viewers generate signals based on a result of sensing operation and / or motions of remote viewers who possess the terminal devices for the remote viewers. The avatar device configures at least one of exercises, vibrations, light emission, sound emission, and image display on the basis of the remote viewers' operation / motion signals.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method for holding a performance event in which a large number of viewers watch a "performance by a player at an event venue".

Background Art

[0002] In order to prevent the spread of the novel coronavirus disease (COVID-19), performance events in which a large number of viewers watch a "performance (game or performance) by a player (athlete or musician) at an event venue", such as sports events and concerts, have been canceled or postponed.

[0003] In order to improve this situation, in performance events, so-called spectator-free games and spectator-free performances are held, and the situation is broadcast live on TV and the Internet.

[0004] However, in a form where viewers of a performance event do not go to the event venue (in the case of a sports event, a "stadium" or "arena", and in the case of a concert, a "concert hall") and watch from a location remote from the event venue, the player cannot perceive in real time the reactions (such as "applause", "cheers", "support", "booing", etc. in the case of a sports event, and "applause", "cheers", etc. in the case of a concert) of viewers located at a location remote from the event venue (hereinafter abbreviated as "remote viewers"), and remote viewers also cannot perceive in real time the reactions of other remote viewers. Therefore, there is a problem that the performance event still lacks excitement.

[0005] Furthermore, in typical performance events, event organizers can collect admission fees from spectators to the event venue (a "stadium" or "arena" in the case of sporting events, or a "concert hall" in the case of concerts). However, in the case of games or performances held without spectators, it is not possible to collect admission fees from remote viewers. This presented a problem where, not only was it impossible to make a profit from holding the event, but it was also impossible to cover the necessary expenses. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] This invention has been made in view of these circumstances, and aims to provide a method for conducting a performance event in which players can perceive the reactions of remote viewers in real time, and remote viewers can also perceive the reactions of other remote viewers in real time.

[0007] Furthermore, the present invention aims to provide a method for conducting a performance event that enables the event organizer to reliably obtain compensation from remote viewers for their attendance at the event. [Means for solving the problem]

[0008] To achieve the above objective, the first aspect of the present invention relating to a method for conducting a performance event is: A method for conducting a performance event in which a large number of spectators watch a performance by players at an event venue, The vast majority of the spectators were located in locations far from the venue of the aforementioned event. Each spectator located in a location far from the venue of the aforementioned event (hereinafter abbreviated as "remote spectator") shall possess a remote spectator terminal device, At the aforementioned event venue, multiple avatar devices, each corresponding to a remote viewer, were installed. The avatar device generates a live video signal based on the results captured by its own imaging means, and generates a live audio signal based on the results collected by its own sound collection means. The remote viewer terminal device displays the live video on its own display means based on the live video signal generated by the avatar device associated with the remote viewer that owns it, and emits live audio on its own sound emission means based on the live audio signal generated by the avatar device associated with the remote viewer that owns it. The remote viewer terminal device generates a signal (hereinafter abbreviated as "remote viewer operation / action signal") based on the results of sensing the operation and / or action (including vocalization) of the remote viewer who possesses the remote viewer terminal device, and the avatar device realizes at least one of motion, vibration, light emission, sound emission, and image display based on the remote viewer operation / action signal generated by the remote viewer terminal device possessed by the remote viewer to which it is associated. It was structured as follows.

[0009] Furthermore, in order to achieve the above objective, a second aspect of the present invention relating to a method for conducting a performance event is: A method for conducting a performance event in which a large number of spectators watch a performance by players at an event venue, The vast majority of the spectators were located in locations far from the venue of the aforementioned event. Each spectator located in a location far from the venue of the aforementioned event (hereinafter abbreviated as "remote spectator") shall possess a remote spectator terminal device, At the aforementioned event venue, multiple avatar devices, each corresponding to a remote viewer, were installed. The avatar device generates a live video signal based on the results captured by its own imaging means, and generates a live audio signal based on the results collected by its own sound collection means. The remote viewer terminal device displays the live video on its own display means based on the live video signal generated by the avatar device associated with the remote viewer that owns it, and emits live audio on its own sound emission means based on the live audio signal generated by the avatar device associated with the remote viewer that owns it. The remote viewer terminal device generates an image signal (hereinafter abbreviated as "remote viewer appearance image signal") based on the result of capturing the appearance of the remote viewer who possesses the remote viewer terminal device, and the avatar device displays the appearance image of the remote viewer based on the remote viewer appearance image signal generated by the remote viewer terminal device possessed by the remote viewer to which it is associated. It was structured as follows.

[0010] Furthermore, in order to achieve the above objective, a third aspect of the present invention relating to a relay device used in a performance event implementation method is: A relay device used in a first or second type of performance event implementation method, The system receives the live video signal and live audio signal from the aforementioned cloning device, performs the necessary processing on the said video signal and audio signal, and then transmits them via the public network to a remote viewer terminal device associated with the cloning device that originated the signal. The remote viewer terminal device receives remote viewer operation / action signals and / or remote viewer appearance image signals via a public network, performs necessary processing on the remote viewer operation / action signals and / or remote viewer appearance image signals, and then transmits them to a avatar device associated with the remote viewer terminal device that originated the signals. It was structured as follows. [Effects of the Invention]

[0011] According to the first embodiment of the present invention, a method for conducting a performance event and a relay device used in the method for conducting a performance event, remote spectators can clap, cheer, and send encouragement while enjoying live video displayed on the display means of their own remote spectator terminal device and live audio emitted from the sound emission means. They can also heckle or boo. The remote spectator terminal device owned by the remote spectator senses these actions (including vocalizations) and generates signals based on the sensed results. Furthermore, while enjoying the live video and live audio, remote spectators can express their emotions such as empathy, dislike, or excitement by operating buttons and levers on their own remote spectator terminal device. The remote spectator terminal device senses these operations and generates signals based on the sensed results. Meanwhile, each of the multiple avatar devices installed at the event venue realizes at least one of motion, vibration, light emission, sound emission, and image display based on the remote spectator operation / action signals generated by the remote spectator terminal device associated with it. As a result, players can perceive the reactions of remote viewers by observing the movement, vibration, light emission, sound emission, and image display of multiple avatar devices installed at the event venue. Remote viewers can also perceive the reactions of other remote viewers by viewing and listening to the live video and audio of the movement, vibration, light emission, sound emission, and image display of the multiple avatar devices installed at the event venue through the display and sound emission means of their own remote viewer terminal devices.

[0012] Furthermore, according to the second embodiment of the present invention, a performance event implementation method and a relay device used in the performance event implementation method, remote viewers can express facial expressions such as laughter or anger, and move their bodies while enjoying live video displayed on the display means of their own remote viewer terminal device and live sound emitted from the sound emission means. The remote viewer terminal device owned by the remote viewer captures an appearance image including these facial expressions and movements, and generates a remote viewer appearance image signal based on the captured results. Meanwhile, each of the multiple avatar devices installed at the event venue displays the appearance image of the remote viewer based on the remote viewer appearance image signal generated by the remote viewer terminal device associated with it. As a result, players can perceive the reactions of remote viewers by observing the appearance images of remote viewers displayed on the multiple avatar devices installed at the event venue. In addition, remote viewers can perceive the reactions of other remote viewers by viewing the appearance images of other remote viewers displayed on the multiple avatar devices installed at the event venue through the display means of their own remote viewer terminal device.

[0013] Furthermore, according to the performance event implementation method and the relay device used in the performance event implementation method of the present invention, live video signals can be transmitted only to remote viewer terminal devices held by remote viewers that are associated with a plurality of avatar devices installed at the event venue. Therefore, by limiting the association with the avatar devices to only remote viewer terminal devices held by remote viewers who have paid or promised to pay the viewing fee in advance, it is possible to reliably obtain payment from remote viewers for viewing the performance event. [Brief explanation of the drawing]

[0014] [Figure 1] This is a block diagram illustrating the configuration and functions of a system for realizing the performance event implementation method according to Embodiment 1 of the present invention, and the functions of a relay device used in the performance event implementation method. [Figure 2] It is an external view for explaining the configuration and functions of a terminal device for remote viewers used in the performance event implementation method according to Example 1 of the present invention. [Figure 3] It is an external view for explaining the configuration and functions of the avatar device used in the performance event implementation method according to Example 1 of the present invention. [Figure 4] It is an external view for explaining the movement of the avatar device used in the performance event implementation method according to Example 1 of the present invention.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to such embodiments, and various modifications are possible within the scope of its technical idea.

Examples

[0016] FIG. 1 is a block diagram for explaining the configuration and functions of a system for realizing the performance event implementation method according to Example 1 of the present invention, and particularly, the functions of a relay device used in the performance event implementation method. It is a block diagram for explaining a situation where remote viewers A to X located at a place remote from the event venue are viewing a concert of a three-member female idol group being held at an outdoor event venue. Also, FIG. 2 is an external view for explaining the configuration and functions of a terminal device for remote viewers used in the performance event implementation method according to Example 1 of the present invention. Also, FIG. 3 is an external view for explaining the configuration and functions of the avatar device used in the performance event implementation method according to Example 1 of the present invention. Also,

[0017] A system for implementing the performance event implementation method of this embodiment is composed of a clone device A_1A to a clone device X_1X, a relay device 2, and terminal devices for remote viewers A to X, namely, a terminal device for remote viewer A_4A to a terminal device for remote viewer X_4X. The relay device 2 and the terminal devices for remote viewers A_4A to X_4X are connected to the Internet.

[0018] Each of the clone devices A_1A to X_1X is composed of a drone 11, an imaging means 12, and a tablet 13. The imaging means 12 and the tablet 13 are fixed to the drone 11.

[0019] The drone 11 is a multi-copter type unmanned aerial vehicle. By controlling the rotation speed of each rotor blade, it can float above a specific area such as a tourist destination and perform straight-line motion, turning motion, and rotational motion.

[0020] In addition, the imaging means 12 includes two omnidirectional cameras that cover a 360-degree viewing angle (solid angle). It captures the performance of the player (in this embodiment, a three-person female idol group), such as singing and dancing, and the situation of other event venues. Furthermore, the imaging means 1 generates an omnidirectional 3D live video signal based on the results captured by the two omnidirectional cameras and wirelessly transmits the omnidirectional 3D live video signal to the relay device 2.

[0021] In addition, the tablet 13 includes a display panel for displaying images, a speaker for emitting sound, and a microphone for collecting the sound of the event venue, including the sound (in this embodiment, singing) emitted by the player (in this embodiment, a three-person female idol group). Also, the tablet 13 generates a live audio signal based on the results captured by the microphone and wirelessly transmits the live audio signal to the relay device 2.

[0022] The relay device 2 accepts viewing applications from remote viewers before the performance event takes place and, based on these applications, associates remote viewers with avatar devices. In this embodiment, each of the avatar devices A_1A to X_1X corresponds to each of the remote viewers A to X in a one-to-one correspondence without any excess or deficiency. However, the number of remote viewers who have applied to view the event may be less than the number of avatar devices installed at the event venue, or multiple remote viewers may share a single avatar device.

[0023] The relay device 2 performs the necessary processing on the full-spherical 3D live video signal received from the imaging means 12 of the avatar device A_1A and the live audio signal received from the tablet 13 of the avatar device A_1A, and then transmits them to the remote viewer A terminal device 4A via the internet 3. Similarly, signals received from avatar devices B_1B to avatar device X_1X are also transmitted to the remote viewer B terminal device 4B to the remote viewer X terminal device 4X.

[0024] On the other hand, each of the terminal devices for remote viewer A_4A to remote viewer X_4X consists of an HMD (Head Mounted Display) 41, headphones 42, a controller 43, a webcam 44, and a signal processing / communication unit 45. The HMD (Head Mounted Display) 41, headphones 42, controller 43, webcam 44, and the signal processing / communication unit 45 transmit and receive signals.

[0025] Meanwhile, the HMD41 is equipped with sensors that detect the head and eye movements of the remote viewer wearing it, and these sensors transmit signals based on the detection results of each sensor to the signal processing and communication unit 45. The signal processing and communication unit 45 sets the display range of the images projected from the left and right projection units, and based on the display range setting result, generates a 3D live video signal from the full-spherical 3D live video signal received from the relay device 2 via the internet 3, and transmits this 3D live video signal to the HMD41. The HMD41 projects images from the left and right projection units based on the 3D live video signal received from the signal processing and communication unit 45. As a result, each of the remote viewers A to X can see the "live video of the performance event" within a field of view corresponding to the movement of their own head and eye.

[0026] The signal processing and communication unit 45 processes the live audio signal received from the relay device 2 via the internet 3 and then transmits it to the headphones 42. The headphones 42 emit sound based on the live audio signal received from the signal processing and communication unit 45. As a result, each of the remote viewers A to X can hear the live audio of the performance event.

[0027] Each of the remote viewers A to X operates the controller 43 while watching the live video and listening to the live audio. The controller 43 is a radio-controlled joystick controller that generates flight control signals based on the operator's lever and dial operations and transmits the flight control signals to the signal processing and communication unit 45.

[0028] The signal processing and communication unit 45 performs the necessary processing on the flight control signal received from the controller 43 and then transmits it to the relay device 2 via the internet 3.

[0029] The relay device 2 performs the necessary processing on the flight control signal received from the signal processing and communication unit 45 via the internet 3, and then transmits the flight control signal to the drone 11 of the avatar device (one of the avatar devices A_1A to avatar device X_1X) associated with the remote viewer who possesses a remote viewer terminal device (one of the remote viewer terminal devices A_4A to remote viewer terminal device X_4X) that includes the signal processing and communication unit 45, which is the source of the flight control signal.

[0030] The rotation speed of the rotor blades of the drone 11 is controlled based on the received flight control signal, and as a result, the drone 11 flies as intended by the remote operator to which the doppelganger device (any of doppelganger devices A_1A to X_1X) is associated, including itself.

[0031] Meanwhile, the webcam 44 captures an image of the remote viewer's appearance, generates a video signal based on the captured image, and transmits the video signal to the signal processing and communication unit 45. The webcam 44 is also equipped with a microphone, which collects sounds such as voices and applause made by the remote viewer, generates an audio signal based on the collected sound, and transmits the video signal to the signal processing and communication unit 45.

[0032] The signal processing and communication unit 45 performs the necessary processing on the video and audio signals received from the controller 43, and then transmits them to the relay device 2 via the internet 3.

[0033] The relay device 2 performs the necessary processing on the video and audio signals received from the signal processing and communication unit 45 via the internet 3, and then transmits the video and audio signals to the tablet 13 of the avatar device (one of the avatar devices A_1A to X_1X) associated with the remote viewer who possesses a remote viewer terminal device (one of the remote viewer terminal devices A_4A to X_4X) that includes the signal processing and communication unit 45, which is the source of the video and audio signals.

[0034] Based on the received video signal, the tablet 13 displays an image of the remote operator, associated with one of the cloning devices (cloning devices A_1A to X_1X), including itself, on its own display panel. Based on the received audio signal, it also emits sounds such as voices and applause from the remote operator, associated with one of the cloning devices (cloning devices A_1A to X_1X), on its own display panel.

[0035] Meanwhile, the signal processing and communication unit 45 determines the psychological state (level of excitement, type of emotion, etc.) of a remote viewer who possesses a remote viewer terminal device (any of remote viewer terminal device A_4A to remote viewer terminal device X_4X) that includes the signal processing and communication unit 45, based on signals received from the HMD 41's sensors based on "movements of the remote viewer's head and eyeballs," and / or video signals and / or audio signals received from the Webcam 44. Based on the determination result, it generates an excitement level signal and an emotion signal, and transmits the excitement level signal and emotion signal to the relay device 2 via the Internet 3.

[0036] The relay device 2 performs the necessary processing on the excitement level signal and the joy, anger, sadness, and happiness signals received from the signal processing and communication unit 45 via the internet 3, and then transmits the excitement level signal and the joy, anger, sadness, and happiness signals to the tablet 13 of the avatar device (one of the avatar devices A_1A to X_1X) associated with the remote viewer who possesses a remote viewer terminal device (one of the remote viewer terminal devices A_4A to X_4X) that includes the signal processing and communication unit 45, which is the source of the transmission of the excitement level signal and the joy, anger, sadness, and happiness signals.

[0037] The tablet 13 changes the brightness of the display panel and the color tone of the displayed image based on the received excitement level signal and emotion signal. Specifically, if the intensity of the received excitement level signal is high, the brightness of the display panel is increased, and if the intensity of the received excitement level signal is low, the brightness of the display panel is decreased. In addition, if an emotion signal corresponding to a positive emotion such as joy or pleasure is received, the image displayed on the display panel will have a warm color tone, and if an emotion signal corresponding to a positive emotion such as anger or sadness is received, the color tone will be cool.

[0038] Furthermore, the relay device 2 performs the necessary processing on the excitement level signal received from the signal processing and communication unit 45 via the internet 3, and then transmits the excitement level signal to the drone 11 of the avatar device (one of the avatar devices A_1A to X_1X) associated with the remote viewer who possesses a remote viewer terminal device (any of the remote viewer terminal devices A_4A to X_4X) that includes the signal processing and communication unit 45, which is the source of the excitement level signal.

[0039] Based on the received excitement level signal, the drone 11 performs one of the following movements, as shown in Figure 4: "axial rotational vibration," "vertical vibration," or "horizontal vibration," or a combination thereof. When the intensity of the received excitement level signal is high, the amplitude of the vibration is increased; when the intensity of the received excitement level signal is low, the amplitude of the vibration is decreased or the vibration (movement) is stopped. [Industrial applicability]

[0040] This invention can also be used in industries that provide services related to performance events such as sports and concerts. Furthermore, it can be used in industries that manufacture servers and other equipment used as relay devices. [Explanation of Symbols]

[0041] 1A‥‥‥Duplicate device A 1B‥‥‥Duplicate device B 1C‥‥‥Duplicate device C 1D‥‥‥Duplicate device D 1E‥‥‥Duplicate device E 1X‥‥‥Duplicate device X 11... Drone 12. Imaging means 13... Tablet 2... Relay device 3. Internet 4A...Terminal device for remote viewer A 4B... Terminal device for remote viewer B 4C... Terminal device for remote viewer C 4D... Terminal device for remote viewer D 4E... Terminal device for remote viewer E 4X... Terminal device for remote viewer X 41...HMD 42... Headphones 43... Controller 44...Webcam 45...Signal Processing / Communication Unit

Claims

1. A method for conducting a performance event in which a large number of spectators watch a performance by players at an event venue, The vast majority of the spectators were located in places far from the aforementioned event venue. Each spectator located in a location far from the aforementioned event venue (hereinafter abbreviated as "remote spectator") shall possess a remote spectator terminal device, At the aforementioned event venue, multiple avatar devices, each corresponding to a remote viewer on a one-to-one basis, and a relay device were installed, The avatar device generates a live video signal based on the results captured by its own imaging means, and generates a live audio signal based on the results collected by its own sound collection means; the remote viewer terminal device displays the live video on its own display means based on the live video signal generated by the avatar device associated with the remote viewer that owns it, and emits live audio on its own sound emission means based on the live audio signal generated by the avatar device associated with the remote viewer that owns it; The remote viewer terminal device generates a signal (hereinafter abbreviated as "remote viewer operation / action signal") based on the results of sensing the operation and / or action (including vocalization) of the remote viewer who possesses the remote viewer terminal device, and the avatar device realizes at least one of motion, vibration, light emission, sound emission, and image display based on the remote viewer operation / action signal generated by the remote viewer terminal device possessed by the remote viewer to which it is associated. The relay device establishes a correspondence between the remote viewer terminal device and the avatar device. The system receives the live video signal and live audio signal from the aforementioned cloning device, performs necessary processing on the live video signal and live audio signal, and transmits them via the public network to a remote viewer terminal device associated with the cloning device that originated the signal. Simultaneously, it receives remote viewer operation / action signals from the remote viewer terminal device via the public network, performs necessary processing on the remote viewer operation / action signals, and transmits them to the cloning device associated with the remote viewer terminal device that originated the signal. A method for conducting a performance event characterized by the following features.

2. A method for conducting a performance event in which a large number of spectators watch a performance by players at an event venue, The vast majority of the spectators were located in places far from the aforementioned event venue. Each spectator located in a location far from the aforementioned event venue (hereinafter abbreviated as "remote spectator") shall possess a remote spectator terminal device, At the aforementioned event venue, multiple avatar devices, each corresponding to a remote viewer on a one-to-one basis, and a relay device were installed, The avatar device generates a live video signal based on the results captured by its own imaging means, and generates a live audio signal based on the results collected by its own sound collection means; the remote viewer terminal device displays the live video on its own display means based on the live video signal generated by the avatar device associated with the remote viewer that owns it, and emits live audio on its own sound emission means based on the live audio signal generated by the avatar device associated with the remote viewer that owns it; The remote viewer terminal device generates an image signal (hereinafter abbreviated as "remote viewer appearance image signal") based on the result of capturing the appearance of the remote viewer who possesses the remote viewer terminal device, and the avatar device displays the appearance image of the remote viewer based on the remote viewer appearance image signal generated by the remote viewer terminal device possessed by the remote viewer to which it is associated. The relay device establishes a correspondence between the remote viewer terminal device and the avatar device. The system receives the live video signal and live audio signal from the aforementioned avatar device, performs necessary processing on the live video signal and live audio signal, and transmits them via the public network to a remote viewer terminal device associated with the avatar device that originated the signal. Simultaneously, it receives a remote viewer appearance image signal from the remote viewer terminal device via the public network, performs necessary processing on the remote viewer appearance image signal, and transmits it to the avatar device associated with the remote viewer terminal device that originated the signal. A method for conducting a performance event characterized by the following features.

3. A relay device used in the performance event implementation method described in claim 1 or 2, The remote viewer terminal device and the avatar device are associated with each other. The system receives the live video signal and live audio signal from the aforementioned cloning device, performs the necessary processing on the video signal and audio signal, and then transmits them via the public network to a remote viewer terminal device associated with the cloning device that originated the signal. The remote viewer terminal device receives remote viewer operation / action signals and / or remote viewer appearance image signals via a public network, performs necessary processing on the remote viewer operation / action signals and / or remote viewer appearance image signals, and then transmits them to a doppelganger device associated with the remote viewer terminal device that originated the signals. A relay device characterized by the following features.

Images