Distribution system

Abstract

To provide a distribution system that gives performers and viewers a sense of realism.SOLUTION: In a distribution system, a distribution device acquires performance video of a performer giving a demonstration and transmits the acquired performance video, a server device receives the performance video transmitted by the distribution device, generates video for distribution on the basis of the received performance video and transmits the generated video for distribution to a receiving device, the receiving device acquires viewer video filmed of a viewer watching the video for distribution transmitted by the server device and transmits the acquired viewer video to the server device, the server device receives the viewer video transmitted by the receiving device and generates viewing status video in which the viewer video is displayed or hidden in a display mode corresponding to the viewing status of the viewer watching the video for distribution and transmits the generated viewing status video to the distribution device, and the distribution device receives the viewing status video transmitted by the server device and displays the received viewing status video on the distribution side display unit.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a distribution system. [Background technology]

[0002] A distribution device for distributing a video of a performer performing a demonstration, etc., and a device for receiving the distributed video. A distribution system including a receiving device that displays the content on a display screen is known (see, for example, Patent Document 1). . [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3832318 Summary of the Invention [Problem to be solved by the invention]

[0004] In the distribution system described in Patent Document 1, it is difficult to give a sense of realism to the performer and the audience. It is difficult.

[0005] The present invention has been made in view of the above, and is intended to provide a performer and an audience with a sense of realism. The object of the present invention is to provide a distribution system that enables the above. [Means for solving the problem]

[0006] The distribution system according to the present invention acquires a performance video of a performer performing a performance, and A distribution device that transmits a performance video, and a receiving device that receives the performance video transmitted from the distribution device. A server that generates a video for distribution based on the demonstration video and transmits the generated video for distribution. a server device for receiving the video to be distributed transmitted from the server device, a receiving device that displays the distribution video on a receiving side display unit, Acquire a viewer image obtained by photographing the viewer watching the program, and transmit the acquired viewer image; The server device receives the viewer video transmitted from the receiving device, and The viewer's video is displayed or hidden in a display mode according to the viewing state of the viewer who is viewing the video. and transmitting the generated viewing status video, The viewing status video transmitted from the server device is received, and the received viewing status video is distributed. The signal is displayed on the receiving side display. Effect of the Invention

[0007] According to the present invention, a distribution system capable of giving a sense of realism to performers and viewers is provided. It is possible. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a distribution system according to the present embodiment. [Diagram 2] FIG. 2 is a functional block diagram illustrating an example of a distribution device. [Diagram 3] FIG. 3 is a diagram illustrating an example of a distribution device and devices connected to the distribution device. [Figure 4] FIG. 4 is a functional block diagram illustrating an example of a server device. [Diagram 5] FIG. 5 is a functional block diagram illustrating an example of a receiving device. [Figure 6] FIG. 6 is a diagram illustrating an example of a video for distribution. [Figure 7] FIG. 7 is a diagram illustrating an example of the viewing status image. [Figure 8] FIG. 8 is a diagram for explaining a setting example of the display mode of the viewer video. [Figure 9]FIG. 9 is a diagram for explaining a setting example of the display mode of the viewer video. [Figure 10] FIG. 10 is a diagram for explaining a setting example of the display mode of the viewer video. [Figure 11] FIG. 11 is a diagram for explaining an example of setting the display mode of the viewer video. [Figure 12] FIG. 12 is a flowchart showing the flow of operations of the distribution system according to this embodiment. [Figure 13] FIG. 13 is a diagram illustrating another example of a distribution device and a device connected to the distribution device. [Figure 14] FIG. 14 is a diagram showing an example of a management screen displayed on the distribution device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of a distribution system according to the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments. includes those which are replaceable and easily understood by a person skilled in the art, or those which are substantially the same.

[0010] FIG. 1 is a diagram illustrating an example of a distribution system 100 according to the present embodiment. As shown, the distribution system 100 includes a distribution device 10, a server device 30, and a receiving device 50. The distribution device 10, the server device 30, and the receiving device 50 are connected via a network NW. The distribution system 100 distributes the performances of performers via a network NW. The video and audio of the performance are distributed to the audience. Music, singing songs, theater, dance, acrobatics, entertainment, recitation, presentations, weddings, fitness Examples include training, lectures, classes, etc.

[0011] FIG. 2 is a functional block diagram showing an example of the distribution device 10. As shown in FIG. The device 10 includes an input unit 11, an output unit 12, a communication unit 13, a control unit 14, and a storage unit 15. Prepare.

[0012] The input unit 11 is connected to input devices such as an image capture unit 16, a voice input unit 17, and an operation unit 18. The image capturing unit 16, the voice input unit 17, the operation unit 18, and the like are all or partly provided in the distribution device 10. FIG. 3 shows a configuration of a distribution device 10 and a device connected to the distribution device 10. 1 is a diagram showing an example of a schematic configuration of the imaging unit 16. The imaging unit 16 has, for example, one or more cameras. The image capturing unit 16 captures the performance of the performer P and generates a performance video. For example, the audio input unit 17 may be arranged on the right, center, or left side of the stage. The audio input unit 17 collects the sound of a performance by a performer and generates a performance sound. The input unit 17 may be, for example, built into the photographing unit 16. The voice input unit 17 includes: An input device that inputs the sound of an electronic musical instrument or audio mixer instead of or in addition to a microphone. The operation unit 18 may be provided with a keyboard, The operation unit 18 is configured with an input device such as a mouse, a touch panel, etc. The operation unit 18 receives predetermined information, for example, a viewing situation displayed on the display unit 19 described later. An input operation can be performed to select one or more viewer images from the video.

[0013] The output unit 12 is connected to output devices such as a display unit (distribution side display unit) 19 and an audio output unit 20. The display unit (distribution side display unit) 19, the audio output unit 20, and other parts or all of the display unit 19 and the audio output unit 20 are controlled by the distribution device 10. The display unit 19 displays information such as images. The display unit 19 may include For example, a projection system such as a projector device and a projection screen, a display panel such as a liquid crystal panel, One or more display units 19 may be provided. The display unit 19 is capable of displaying a viewing status image, which will be described later. In this embodiment, the display unit 19 is The size of the exhibition space is designed to cover the entirety of the demonstration space 25 where the performers will be performing. For example, you can mobilize participants in some spaces and display them in empty spaces. The audio output unit 20 outputs audio information. The audio output unit 20 For example, it is composed of a speaker.

[0014] The communication unit 13 is capable of transmitting and receiving information via the network NW. The communication unit 13 may be a unit for performing either wired communication or wireless communication. It transmits and receives information to and from device 30.

[0015] The control unit 14 controls the processing of a CPU (Central Processing Unit) and the like. Device and RAM (Random Access Memory) or ROM (Read The control unit 14 controls the distribution device 10. The control unit 14 includes an input information acquisition unit 21, a communication control unit 22, and an output control unit 23. and a response information acquisition unit 24.

[0016] The input information acquisition unit 21 acquires information such as a performance video, a performance audio, and an operation signal input to the input unit 11. The communication control unit 22 acquires various pieces of information received by the communication unit 13.

[0017] The communication control unit 22 transmits the performance video and performance audio acquired by the input information acquisition unit 21 from the communication unit 13. The communication control section 22 causes the responding information of the performer, which will be described later, to be transmitted to the server device 30. 13 to the server device 30.

[0018] The output control unit 23 controls the operation of the display unit 19 and the audio output unit 20. The output control unit 23 receives the display contents from the server device 30 (to be described later). The transmitted viewing status video is displayed on the display unit 19.

[0019] The response information acquisition unit 24 acquires the response information of the performer to the viewing situation image displayed on the display unit 19. The response information is, for example, a response from the performer to the viewer image displayed in the viewing image. Information indicating that a response has been made and the audience (or receiving device) to whom the performer responded 50) and information indicating the

[0020] The storage unit 15 stores various information. The storage unit 15 is, for example, a hard disk drive, The storage unit 15 includes a solid state drive. An external storage medium such as a bubble disk may also be used.

[0021] FIG. 4 is a functional block diagram showing an example of the server device 30. As shown in FIG. The server device 30 includes a communication unit 31, a control unit 32, and a storage unit 33. The communication unit 31 Information can be communicated between the distribution device 10 and the receiving device 50 via the network NW. .

[0022] The control unit 32 comprehensively controls the operation of the server device 30. The control unit 32 includes a CPU, etc. The control unit 32 includes a processing unit and a storage unit such as a RAM or a ROM. The video data generating unit 35 has a video data for distribution, a viewing status video data generating unit 36 ​​, and a volume setting unit 37 .

[0023] The communication control unit 34 controls the operation of the communication unit 31. The communication control unit 34 also receives various information from the communication unit 31. Can.

[0024] The distribution video generation unit 35 generates a distribution video based on the demonstration video transmitted from the distribution device 10. The video to be distributed is an image to be displayed on a display unit 59 of the receiving device 50, which will be described later. The distribution image generating unit 35 may be configured to be included in the distribution device 10. The trust image generating unit 35 may be configured to be included in the receiving device 50. In this case, the receiving device 5 0 generates an image (image for distribution) to be displayed on the display unit 59 based on the demonstration image. .

[0025] The viewing status image generating unit 36 ​​generates a viewing status image based on the viewer image transmitted from the receiving device 50. A viewing status image is generated. The viewing status image indicates the viewing status of the viewer who is viewing the distribution image 70. The viewing status image is a display according to the viewing status of the viewer who is viewing the distribution image 70. The image is an image in which the viewer image is displayed or not displayed in a display mode.

[0026] The volume setting unit 37 sets the volume of the performance sound and the viewer sound, which will be described later.

[0027] The storage unit 33 stores various information. The storage unit 33 may be, for example, a hard disk drive, The storage unit 33 includes a solid state drive. Alternatively, an external storage medium such as a portable disk may be used. The data of the video to be distributed that is to be displayed on the display unit of the device 50 is stored.

[0028] FIG. 5 is a functional block diagram showing an example of a receiving device 50. As shown in FIG. The device 50 includes an input unit 51, an output unit 52, a communication unit 53, a control unit 54, and a storage unit 55. The receiving device 50 may be, for example, a smartphone, a tablet, or a personal computer. The receiving device 50 is a general-purpose terminal such as a personal computer, a karaoke terminal, etc. The receiving device 50 may be a terminal owned by a viewer or a terminal provided by a distribution company. Owned by a business operator who manages facilities such as karaoke stores (facilities equipped with receiving device 50) The terminal device 50 owned by the broadcasting company may be a terminal that the broadcasting company lends to the viewer. The equipment may be used by the public or by facilities such as karaoke shops managed by businesses (facilities equipped with the receiving device 50). In the distribution system 100, the receiving device 50 includes: One or more, for example, a plurality, are provided.

[0029] The input unit 51 inputs various information. The input unit 51 includes an image capture unit 56, a voice input unit 57, The imaging unit 56, the voice input unit 57, the operation unit 58, etc. may be connected to the imaging unit 56, the voice input unit 57, the operation unit 58, etc. The image capturing unit 56 may be, for example, one or more cameras. The camera 56 captures the viewer watching the video to be distributed and generates a viewer video. The voice input unit 57 has, for example, one or more microphones. The voice of the viewer watching the credit video or the sound of applause is collected to generate the viewer voice. The voice input unit 57 may be configured to be built into the photographing unit 56. For example, the operation unit 58 is configured with input devices such as a keyboard, a mouse, and a touch panel. The operation unit 58 inputs predetermined information to the receiving device 50. For example, the position of the viewer's icon This can be used to set

[0030] The output unit 52 is connected to a display unit (receiving side display unit) 59, an audio output unit 60, etc. (Receiving side display unit) 59, a part or all of the audio output unit 60, etc. are provided in the receiving device 50. The display unit 59 displays information such as images. The display unit 59 is, for example, a liquid crystal panel. The display unit 59 has a display panel such as a display unit. The display unit 59 is capable of displaying a distribution video 70. The display unit 59 of the 50 displays a user interface (application) in addition to the distribution video 70. The user interface may be a dedicated application software operation screen. It can be a native application or a program that runs in a web browser. The display unit 59 may be an application that displays a voice message. The output unit 20 outputs audio information. The audio output unit 20 is configured with, for example, a speaker. can be.

[0031] The communication unit 53 is capable of transmitting and receiving information via the network NW. The communication unit 53 may be a unit for performing either wired communication or wireless communication. It transmits and receives information to and from device 30.

[0032] The control unit 54 includes a processing device such as a CPU and a storage device such as a RAM or a ROM. The control unit 54 performs overall control of the receiving device 50. 1, a communication control unit 62, and an output control unit 63.

[0033] The input information acquisition unit 61 acquires information such as the viewer's video, audio, and operation signals input to the input unit 51. The communication control unit 62 acquires various information received by the communication unit 53. The communication control unit 62 transmits the video and audio of the viewer acquired by the input information acquisition unit 61 from the communication unit 53. To make someone believe.

[0034] The output control unit 63 controls the operation of the display unit 59 and the audio output unit 60. The output control unit 63 controls the content to be displayed on the display unit 59. The output control unit 23 sets the display mode of the video of the person. The display mode of the viewer's video is set based on the above. Display of the viewer's video by the output control unit 23 A specific setting operation of the mode will be described later.

[0035] The storage unit 55 is a storage device such as a hard disk drive or a solid state drive. The storage unit 55 may be an external storage medium such as a removable disk. The storage unit 55 stores various types of information.

[0036] Next, an example of the operation of the distribution system 100 configured as described above will be described. The video input unit 6 and the audio input unit 17 respectively capture images and sounds of a given performance performed by a performer, and The video and audio of the performance are generated. The performance by the performer is, for example, divided into three parts: right side, center side, left side, The imaging may be performed by a plurality of imaging units 16 arranged in different directions, such as on either side of the imaging unit 1. The performance video captured by the video input unit 6 and the performance audio picked up by the audio input unit 17 are input to the distribution device 10. The signal is input to the input unit 11.

[0037] In the distribution device 10, the input information acquisition unit 21 receives the demonstration video and The performance sound acquired by the input information acquisition unit 21 is acquired by the control of the communication control unit 22. The video and the audio of the performance are transmitted from the communication unit 13 to the server device 30. The live performance video and live performance audio transmitted from the receiving device 10 are received.

[0038] The distribution video generation unit 35 generates a distribution video based on the demonstration video transmitted from the distribution device 10. FIG. 6 is a diagram showing an example of a video for distribution. As shown in FIG. 6, The video for distribution 70 includes a display area 71 and a background image 75 in which an icon arrangement area 72 is provided. The display area 71 is an area for displaying a demonstration video 73 transmitted from the distribution device 10. The display area 71 corresponds to, for example, a stage at a concert venue. The icon arrangement area 72 is The icon arrangement area 72 is provided in correspondence with the display area 71. The icon arrangement area 72 is an area where an icon 74 of a person can be arranged. It corresponds to the audience seats. The distribution image 70 is a so-called Virtual Reality image. The distribution image generating unit 35 generates the actual image transmitted from the distribution device 10. The display unit 59 of the receiving device 50 has a plurality of For example, large-screen displays and smartphones, tablets, and The distribution image generating unit 35 may be divided into a karaoke terminal and a display unit 59. For example, the display area 71 is displayed on a large screen display, and the icon arrangement area is smartly arranged. Even if you generate distribution video to be displayed on a smartphone, tablet, or karaoke terminal, In the distribution video 70, the user can drag his / her own icon 74 on the operation unit 58 of the receiving device 50. When you drag or swipe, you can move the position of your icon 74. If you drag or swipe anywhere other than the icon 74, the image display angle will change. Or you can change the range. Also, you can pinch in and pinch out the area on the touch panel. The display range and magnification of the distribution image 70 can be adjusted by the down operation or the scroll operation, respectively. For example, if you want to display only the demonstration video 73, By operating the device, the size of the demonstration image 73 is made the same as the size of the display unit 59 of the receiving device 50. In addition, when such an operation is performed, the display unit 59 of the receiving device 50 displays The sound from the icon 74 that has disappeared or the demonstration image 73 is not displayed on the display unit 59. When the number of times the live performance video 73 is changed, the live performance audio is not output from the audio output unit 60. Good too.

[0039] In addition, the viewer accesses the server device 30 from the receiving device 50 and selects the icon of the background image 75. The icon 74 is set to be placed at a desired position in the icon placement area 72. The image generating unit 35 places the icon at a set position in the icon arrangement area 72 of the background image 75. Combine 74.

[0040] The distribution image generating unit 35 synthesizes the demonstration image 73 on the display area 71 of the background image 75, An icon 74 is composited into the icon placement area 72 to generate a distribution video 70. In the example shown in FIG. 6, for example, six icons 74 (74a to 74f) are combined to form a distribution image. In the following description, when distinguishing between the multiple icons 74 shown in the figure, In this case, a symbol indicating each icon 74 is added, such as icon 74a, icon 74b, etc. The performance by the performer is photographed by a plurality of photographing units 16 arranged in different directions. In this case, the distribution image generating unit 35 performs the following operation based on the positional relationship between the display area 71 and the icon 74: The user can select a demonstration video to be composited with the distribution video 70 from among a plurality of demonstration videos. For example, when the icon 74 is placed on the right side of the display area 71, the distribution image generating unit 35 The icon 74 is transmitted to the receiving device 50 for delivery of the video 70, and the icon 74 is displayed by the performer. A demonstration video 73 of the performance being shot from the right side can be synthesized. Regarding which demonstration image to select from among the multiple demonstration images 73, the icon arrangement area 72 is A selection area is divided into a plurality of areas, and the icon 74 is selected based on which selection area it is located. Based on this, the distribution image generating unit 35 may select the demonstration image 73. The unit 35 may enlarge or reduce the demonstration image 73 based on the position of the icon 72 . In this case, the distribution image generating unit 35 may, for example, adjust the horizontal direction of the demonstration image 73 to the position of the icon 72. The vertical direction of the demonstration image 73 can be enlarged or reduced with the performer's face image at the center. In addition, the distribution image generating unit 35 may determine, based on the relationship between the display area 71 and the icon 74, The image is processed so that the demonstration image 73 changes smoothly in accordance with the movement of the icon 74. In this case, it is preferable that the distribution image generating unit 35 limit the number of the image capturing units 16 to about several tens. It is preferable to use augmented images or perform image processing using 3D modeling, etc. In addition, the distribution image generation unit 35 does not change the position of the icon 74, but instead uses a predetermined icon ( For example, when a viewer clicks or taps on an icon that shows a zoom operation such as binoculars In addition, the distribution image generating unit 35 may switch to an enlarged image of the demonstration image 73. The timing (time) at which a specific icon was clicked or tapped is associated with each viewer and recorded. This table shows that viewers with the same or similar (close) timing have good compatibility. The display may be added to the icon 74. For example, a percentage (a number showing the degree of compatibility) can be displayed as an icon to indicate the compatibility. Alternatively, a video may be generated to be displayed together with the icon 74 of a compatible viewer. You can also create a larger version of the video and encourage interaction with other viewers. The image may be generated so as to be displayed as the above.

[0041] In addition, the volume setting unit 37 determines the position of the display area 71 and the icon 74 in the distribution video 70. Based on the positional relationship, the volume of the live performance sound can be set. For example, the volume setting unit 37 The closer an icon 74 is placed to the display area 71, the greater the number of recipients corresponding to that icon 74. It is possible to set the volume of the live performance sound transmitted to the receiving device 50 to be louder. In addition, the volume setting unit 37 determines whether the icon 74 is placed farther away from the display area 71 than the icon 74. The volume of the live performance sound transmitted to the receiving device 50 corresponding to the icon 74 is set to be low. This operation allows you to hear the sound in the same way as if you were watching a live performance at a live concert. For the same purpose, the volume setting section 37 is provided on the right side of the display area 71. Depending on the left / right position, such as the left, center, left, etc., As a specific example, the volume setting unit 37 may adjust the volume when the icon 74 is When the person in charge is located to the right of the stage, the volume of the instrument being played on the right side may be increased. The setting unit 37 may set not only the volume but also the frequency characteristics. By adding information indicating the direction of the sound and providing directionality to the sound, the volume setting unit 37 can set the icon 74 The volume, audio components, frequency characteristics, etc. may be changed depending on the direction.

[0042] The generated distribution video 70 and live performance audio are transmitted to the receiving device 5 under the control of the communication control unit 34. It will be delivered at 0.

[0043] The receiving device 50 receives the video 70 for distribution transmitted from the server device 30. Communication Control The output control unit 62 acquires the received video for distribution 70. The output control unit 63 outputs the acquired video for distribution The image 70 is displayed on the display unit 59. The viewer can view the distribution image 70 displayed on the display unit 59. It can be viewed.

[0044] The image capturing unit 56 captures images of the viewer viewing the video for distribution 70, and generates a viewer image. The voice input unit 57 collects the voice of the viewer viewing the distribution video 70 and generates the viewer voice. The viewer's video and audio are input to the input unit 51. The input information acquisition unit 61 The input viewer video and viewer audio are acquired. The viewer video and viewer audio are transmitted from the communication unit 53 to the server device 30 .

[0045] The server device 30 receives the viewer's video and the viewer's audio. The communication control unit 34 also The received viewer voice is transmitted to a receiving device 50 different from the receiving device 50 of the receiving destination, for example. This operation allows you to hear the cheers of other spectators when watching a live performance at a venue. In the same way, viewers can share the reactions of other viewers to the distribution video 70. In this case, the volume setting unit 37 can select the icon to be placed in the icon placement area 72. The volume of the viewer's voice can be set based on the relative positions of the 74. The amount setting unit 37 determines whether the icons 74 are closer to each other. The volume of the viewer's voice transmitted to the corresponding receiving device 50 can be set to be louder. In addition, the volume setting unit 37 can set the volume of the icons 74 to be closer to each other as the icons 74 are arranged farther apart. The volume of the viewer's voice transmitted to the receiving device 50 corresponding to the icon 74 of the other party becomes smaller. It can be set as follows.

[0046] The distribution image generating unit 35 displays the acquired viewer image in place of the icon 74 or In addition to the icon 74, the icon can be synthesized in the icon arrangement area 72. The distribution video 70 is transmitted to each receiving device 50 with the viewer video displayed. The video of one viewer is distributed to other viewers. Just as viewers can see what other spectators are doing when watching a live broadcast, viewers can also share the broadcast footage This action also allows viewers to see what other viewers are doing before the demonstration begins. During breaks, or after the performance, viewers who are located near icon 74 can meet with each other. The position of the icon 74 of the user can be changed by dragging the icon 74. The image may be moved by an operation such as "and drop" or by changing the position or direction of the smartphone. It may be possible to use a sensor that acquires such information to move in accordance with the operation of the smartphone. Buttons for moving forward, backward, right, etc. may be provided, or other user interfaces may be used as appropriate. You may use it as appropriate.

[0047] The viewing status image generating unit 36 ​​generates, for example, a combination of the received viewer image and viewer voice and an icon arrangement. Based on the position of the icon 74 set in the placement area 72, the viewer who is to watch the distribution video 70 is The viewing status image generating unit 36 ​​detects the viewing status of the listener and generates a display mode according to the viewing status. The viewing situation image generating unit 3 generates a viewing situation image in which the viewer image is displayed or not displayed. 6 is a display to a viewer whose own viewer image is displayed in the viewing status image as a viewing image. Specifically, the viewing status image generating unit 36 ​​may notify the receiving device 50 that the viewing status image has been The manner in which the icon 74 is displayed on the receiving device 50 may be changed, or a notification may be given to the receiving device 50 by using text, an image, or the like. The viewing status image generating unit 36 ​​may also generate a viewer image to be displayed on the receiving device 50. Before displaying the content, the viewer may confirm that the content is to be displayed. The public range of the video and audio can be set, and the distribution video generation unit 35 uses the set public range Based on this, the acquired viewer image may be synthesized in the icon arrangement area 72. The device 50 may allow a viewer to register other viewers as friends. For example, you can select the icon 74 of another viewer who is not registered as a friend. Move your icon 74 within a specified distance from the 4. Tap or click to send a friend request to viewers who are not registered as friends. If the viewer who received the friend request accepts the request, they will become friends with each other. The range of disclosure that can be set on the receiving device 50 is, for example, viewing status video, friend Regarding the viewer images on the receiving devices 50 of our friends and the viewer images on the receiving devices 50 of viewers other than our friends , each of which can be set to public or private.

[0048] FIG. 7 is a diagram showing an example of a viewing status image. As shown in FIG. The image 80 includes one or more viewer images 81. The viewer images 81 are received by each receiving device 50. The viewing situation image generating unit 36 ​​generates one of the following images according to the viewing situation of the viewer: The above viewer images 81 can be arranged at predetermined positions. In FIG. This shows a case where listener images 81 (81a to 81f) are displayed. When distinguishing between the multiple viewer images 81 shown in the figure, viewer image 81a, viewer image 81b In this case, the viewer image 8 1a is an image of a viewer corresponding to the icon 74a of the distribution image 70 shown in FIG. 6 etc. Also, the viewer image 81b is the viewer's image corresponding to the icon 74b of the distribution image 70. Similarly, viewer images 81c, 81d, 81e, and 81f are respectively distributed The images of the viewers corresponding to the icons 74c, 74d, 74e, and 74f of the trust image 70 are shown. Let us assume that.

[0049] The viewing situation image generating unit 36 ​​may, for example, determine the position of the display area 71 and the position of the icon 74. Based on the positional relationship, the display mode of the viewer's video can be set.

[0050] FIG. 8 is a diagram for explaining a setting example of the display mode of the viewer video. As shown in FIG. The situation image generating unit 36 ​​detects that the icon 74 is located in the vicinity of the display area 71 in the icon arrangement area 72. The viewers located in the vicinity area 72a (corresponding to the icons 74a, 74c, and 74f) For the viewers, the viewing status image 80 shows the viewer images 81 (81b, 81d, 81e). 7, the viewer whose icon 74 is not located in the vicinity area 72a (icons 74b and 74 For viewers corresponding to 74d and 74e, the viewer image is not displayed in the viewing status image 80. In FIG. 8, the icon arrangement area 72 is displayed as a seat. However, the viewing situation image generating unit 36 ​​may display, for example, a free space instead of a seat. Also, the right to watch the live performance video 73 can be used in the same way as a reserved seat ticket for the actual performance venue. The viewing situation image generating unit 36 ​​uses the icon when displaying the display area 71 as the right to reserve a seat. It is possible to position the icon 74 only in the designated seat of the viewer in the icon placement area 72. In this case, the seats in the front can be charged higher. It is acceptable to use images of seats that mimic the seats shown in Figure 8 or images of seats in an actual hall, etc. This allows viewers to enjoy the atmosphere of the venue in addition to the live video. The image generating unit 36 ​​generates a viewing situation image of the viewer whose icon 74 is located in the nearby area 72a. By generating the image to be displayed on the image 80, the viewing experience from the front seats is the same as if the viewer actually went to the venue. This allows the performer to see only the person in the room, providing a sense of realism to both the performer and the audience. Also, the viewing situation image generating unit 36 ​​can provide only the viewers in the front seats. By generating the images to be displayed, it is possible to create a sense of realism for both the performer and the audience, even with limited resources. While providing the viewer with the above-mentioned information, the viewer image 81 can be displayed on the viewing situation image 80. The resources provided include, for example, the display area of ​​the viewing situation image 80, the processing capacity of the server device 80, The viewing status image 80 indicates various restrictions such as the communication load imposed on the network NW. The image may include not only the viewer image 80 but also the image of the icon 74 and the avatar of the viewer. The listening situation image generating unit 36 ​​may be configured to move the icon 74 around the venue. The surrounding area of ​​the venue includes, for example, the lobby, shop, or waiting room of the venue, or Examples include restaurants and other establishments near the demonstration venue. Icon 74 between the demonstration venue and its surroundings The movement of the user may be performed through an image of a doorway or through a three-dimensional virtual space. The viewing status image generating unit 36 ​​may be configured to be able to move the icon 74. Viewer images 81 may be displayed in the viewing status image 80 only for viewers who are in the performance hall. The viewing situation image generating unit 36 ​​generates the image of the live performance so that the audience can concentrate on the sound of the live performance. may automatically reduce the volume of other viewers' audio to a specified value or set it to zero. Even in this case, the viewing situation image generating unit 36 ​​is required to synchronize the viewing situation with the performer with limited resources. The viewer image 81 can be displayed on the viewing situation image 80 while providing a sense of realism to both the listeners and the listeners. In addition, the viewing situation image generating unit 36 ​​recognizes the contents of the conversation between the viewers and converts the conversation into a Content related to the speaker is extracted from the topic of the talk and displayed as text around icon 74. In this case, after the performance, a large number of viewers may share topics related to the performer and have a conversation. It becomes easier to talk.

[0051] FIG. 9 is a diagram for explaining a setting example of the display mode of the viewer video. As shown in FIG. The situation image generating unit 36 ​​performs up / down switching between the position of the display area 71 and the positions of the icons 74a to 74f. The viewer image 81a is inserted in the viewing situation image 80 so that the positional relationship between the left and right sides is the same. This allows the display of 81f to be displayed from the stage side in an actual demonstration venue. The viewer image 81 can be displayed in the same manner as when viewed from the audience seating side.

[0052] FIG. 10 is a diagram for explaining an example of setting the display mode of the viewer video. As shown in FIG. The viewing status image generating unit 36 ​​is a viewer whose icon 74 is placed near the display area 71. The viewer's image 81 is displayed relatively large, and the icon 74 is displayed far from the display area 71. The viewer image 81 of the viewer can be displayed relatively small. The viewer images 8 corresponding to the icons 74c and 74f arranged in the positions closest to the display area 71 are 1c and 81f are displayed largest in the viewing situation image 80. The viewer image 81a corresponding to the icon 74a arranged in the second closest position to the viewer image 81a indicates the viewing state. The second largest display in the status image 80. The third closest display area to the display area 71. The viewer images 81d and 81e corresponding to the icons 74d and 74e arranged in It is displayed as the third largest in the image 80. It is also located at the farthest position from the display area 71. The viewer image 81b corresponding to the icon 74b displayed is the smallest in the viewing status image 80. This allows the image to be displayed in a small size, so that when you look at the audience seats from the stage at an actual concert venue, The viewer image 81 can be displayed in the same manner as above. It can be either still or video, and you can dynamically switch between still images or between still and video as needed. The position of the icon 74 in the virtual space and the position of the viewer image 81 seen by the performer may be By linking the two, it is possible to give the performer a sense of realism. For example, "I want to go to the front," "I want the performers to remember my face," "I'm not good at "There is a performer I support," "I'm embarrassed to be seen by the performer," etc. The viewer can intuitively input the information to the system. By dynamically changing the viewer image 81 while responding to the diverse needs of listeners, The number of viewer videos 81 can be increased or decreased. By dynamically changing the viewer image81 while responding to the diverse needs of the viewer, It also allows viewers to pay in stages, depending on the amount of the charge. The display mode of the viewing status image 80 may be changed accordingly.

[0053] In addition, the viewing situation image generating unit 36 ​​may generate a video of the viewing situation according to the position of the performer P1 relative to the display unit 19. The display mode of the viewer's video can be set accordingly.

[0054] FIG. 11 is a diagram for explaining an example of setting the display mode of the viewer video. As shown in FIG. The viewing situation image generating unit 36 ​​generates an image of the area of ​​the display unit 19 corresponding to the position where the performer P1 is present. In this embodiment, the viewing situation image can be generated so that the viewer image 81 is displayed. This makes it easier for the performer to grasp the viewer image 81. In the case where the listener can select a specific performer, the viewing situation image generating unit 36 ​​displays the The area corresponding to the position of the performer selected by the viewer is the viewing area of ​​the viewer who selected the performer. The viewing situation image can be generated so that the viewer image 81 is displayed. A viewer's image 81 can be displayed near the performer selected by the viewer. Specifically, the distribution device 10 or the server device 30 performs face detection and gaze detection using the image capture unit 16. detection, skeleton detection, etc., or radiation thermometer or ToF (Time of Flight) sensor The position of the performer may be identified using other sensors such as a viewing sensor. , and generates a viewer image 81 based on the position of the performer. When an image or the like that supports the support is posted, the viewing status image generating unit 36 ​​may The system uses image recognition to display the viewer image 81 in a position that is easy to see for the performer being cheered on. The viewing status image generating unit 36 ​​records the detection of the cheering image, Or, the voices of the cheering are recognized and a database is created that associates each viewer with the performer they are cheering for. By creating a user-generated video database, viewers who are supporting a particular performer can watch the video81 of the specific performer. The viewing situation image generating unit 36 ​​may generate an image to be displayed in a position that is easily visible to the performer. The viewer image 81 may be generated so as to dynamically change in accordance with the movements of the performer.

[0055] The viewing situation image generated as described above is transmitted from the communication unit 31 under the control of the communication control unit 34. The viewing status video is then transmitted to the distribution device 10. The distribution device 10 receives the transmitted viewing status video. The reception control unit 22 acquires the received viewing status video. The output control unit 23 outputs the acquired viewing status video. The status image is displayed on the display unit 19.

[0056] The viewing status image generating unit 36 ​​also calculates the degree of viewer enthusiasm for the distribution image 70. The display mode of the viewer's video can be set according to the degree of enthusiasm shown by the viewer. When watching video 70, for example, people are cheering on the performers, and the volume of the cheers is loud. , large movements, moving in response to the movements of the performer, dancing, performer support goods The viewers' faces are smiling, moved to tears (emotionally moved), In the case of a person showing positive behavior toward the distribution video 70, such as being shown in the viewer video, The control unit 32 performs image recognition, face recognition, and other operations on the viewing situation video. Combining voice recognition, skeletal recognition, etc. appropriately, or using radiation thermometer or ToF (Time Other sensors, such as a Flight of Flight sensor, may be used to measure viewer engagement. In addition, the degree of enthusiasm is determined by the viewer's viewing status video, which is allowed to be displayed in the viewing status video. There is no history of the viewer using negative or prohibited words regarding the distribution video 70. , the value is high when the viewer does not behave negatively toward the distribution video 70. Including the above example, the control unit 32 calculates a score for each predetermined item and performs the following operations according to the score. The viewing situation image generating unit 36 ​​may set the degree of enthusiasm by, for example, The viewer image 81 of the viewer with the highest level of enthusiasm is displayed, and the viewer image 82 of the viewer with the lowest level of enthusiasm is displayed. The control unit 32 can generate the viewing situation image 80 so that the image 81 is not displayed. The degree of enthusiasm may be calculated based on the image recognition results of the viewing situation video, or the viewer may post comments in the comment section. The degree of enthusiasm may be calculated based on the content of the text comment entered in the Viewers' SNS (Social Networking Service) accounts The control unit 32 may be configured to link with the SNS, and calculate the degree of positivity by referring to information on the SNS. In addition, when the distribution system 100 is used for school classes, etc., The configuration section 36 deliberately prioritizes viewers who are calculated to have a low level of enthusiasm, such as sleeping students. The viewer video may be generated so that the viewer's video is

[0057] 14 is an example of a management screen that is checked by an operator of the distribution device 10. In the example screen, the management status of users 1 to 30, the forced exit button, and the brightness of the display unit 19 are displayed. The user management status includes the display name , whether the viewer's image 81 is being displayed, whether the viewer's face is displayed in the viewer's image 81, N It also includes the number of times the G word was uttered. It also includes the smile score, cheer score, movement score, and response score. The support goods score is displayed, and the total value of these is used to determine the level of enthusiasm. In the example, the degree of positivity is calculated as the sum of the scores, but it may be calculated using a predetermined coefficient, etc. In the example of the management screen shown in Figure 14, the management status of users 1 to 30 is displayed. However, only the management status of the user who is viewing the video, or the score and status It is also possible to display only the management status of users who have had changes. The score for each item can be calculated from the management screen. Alternatively, the control unit of the server device may automatically perform this process without displaying the information on the screen.

[0058] The performer can respond to the viewing situation image 80 displayed on the display unit 19. The response information acquisition unit 24 receives responses from the performer to the viewing situation image displayed on the display unit 19. When the performer performs the action, the response information of the performer is acquired. In this case, the response information acquisition unit 24 may detect the detected signal and acquire the response information based on the detection result. For example, the line of sight of the performer can be detected based on the performance video captured by the camera unit 16. The response information acquisition unit 24 determines whether the performer is gazing at the display unit 19 based on the detection result. If the person is gazing at the display unit 19, which part of the display unit 19 the person is gazing at is detected. The response information acquisition unit 24 detects whether the performer's line of sight is on the display unit 19 where the viewer's image is displayed. If it is detected that the performer is gazing at the part of the image that is being viewed or its vicinity, In this case, the response information acquisition unit 24 determines that the performer is responding to the gaze of the The viewer image captured at the position where the viewer is viewing the image and in its vicinity is extracted, and the extracted viewer image is processed. In this way, the response information acquisition unit 24 identifies the viewers (or the receiving devices 50) who are to receive the performance. The response information includes information indicating that the performer is responding and information indicating the audience to whom the performer is responding. Obtain as.

[0059] An example of a performer responding to the viewer's image is limited to directing his / her gaze toward the viewer's image. For example, the performer may call out to the viewer, wave, point, or give a glance. The performer or the operator of the distribution device 10 clicks or taps on the viewer image using the operation unit 18. In the example of FIG. 3, the operation unit 18 is a control unit for the distribution device 10. The illustration is based on the mouse used by the author, but the operating unit 18 used by the performer is: For example, a button was added to the microphone so that it could be used during demonstrations, or it was a portable device. The response information acquisition unit 24 may be a terminal capable of receiving, for example, a performance video and a performance audio. Based on the voice, the performer called out to the viewer and the image of the viewer to whom the performer called out The response information acquisition unit 24 acquires, as response information, Waving, pointing, or looking at something, and the viewing or listening behavior of the person who is the target of these actions The response information acquisition unit 24 also acquires the video of the performer as response information. If the viewer selects a video from the list, the viewer will be informed that they have responded by selecting the video and that the video is the one that is the subject of the selection. The viewer's video is acquired as response information.

[0060] The communication control unit 22 transmits the response information acquired by the response information acquisition unit 24 from the communication unit 13 to the server. The server device 30 receives the response information transmitted from the distribution device 10. The communication control unit 34 acquires the response information.

[0061] The server device 30 changes the display mode of at least a part of the video to be distributed based on the response information. For example, the distribution image generating unit 35 may Regarding the distribution video 70 to be transmitted to the receiving device 50 of the viewer, the icon 74 representing the viewer is displayed. The display mode can be changed. For example, the distribution image generating unit 35 can change the display mode by turning on the icon 74. The display changes over time, such as dimming, enlarging or reducing, moving, deforming, discoloring, or a combination of these. It may also be written as:

[0062] By this operation, the display unit 59 of the receiving device 50 of the audience to whom the performer is responding shows the following: The icon 74 is displayed in a changed display state. By visually checking the change in the state of the voice, the user can know that the performer has responded. In the distribution system 100 according to the present invention, a viewer This allows for two-way communication between the performer and the audience. It is possible.

[0063] FIG. 12 is a flowchart showing the flow of operations of the distribution system 100 according to this embodiment. As shown in FIG. 12, the distribution system 100 includes a distribution device 10, a server device 30, The operation is started by the start of the receiving device 50, or the start of shooting or sound collection, etc. The image capturing unit 16 and the audio input unit 17 capture and record the performance by the performer. The generated performance video and performance audio are generated (step S101). The performance audio is input to the input unit 11. The communication control unit 22 controls the input performance video and The voice is transmitted from the communication unit 13 to the server device 30 (step S102). 30 receives the performance video and performance audio transmitted from the distribution device 10 (step S10 3) The distribution image generating unit 35 synthesizes the demonstration image 73 in the display area 71 of the background image 75. (Step S104).

[0064] On the other hand, the viewer accesses the server device 30 from the receiving device 50 and changes the location of the icon 74. (Step S105). The distribution video generating unit 35 of the server device 30 The icon 74 is then composited at the position where the icon 74 was placed (step S106). The communication control unit 22 transmits the generated distribution video 70 to the receiving device from the communication unit 31. The information is transmitted to device 50 (step S107).

[0065] The receiving device 50 receives the distribution video 70 transmitted from the server device 30 (step The received distribution video 70 is acquired by the communication control unit 62 and is then output to the output control unit 6 The image displayed on the display unit 59 is displayed by the display unit 59 via the display unit 3 (step S109). The user can watch the distribution video 70 that will be distributed.

[0066] The image capturing unit 56 and the audio input unit 57 capture images of viewers watching the distribution video 70 and The viewer's image and the viewer's voice are collected, and a viewer image and a viewer's voice are generated (step S110). The image and the viewer's voice are input to the input unit 51, and are transmitted from the communication unit 53 under the control of the communication control unit 62. The information is then transmitted to server device 30 (step S111).

[0067] The server device 30 receives the viewer's video and viewer's audio (step S112). The situation video generating unit 36 ​​detects the viewing state of the viewer who is viewing the distribution video 70, and A viewing status image is generated in which the viewer image is displayed or not displayed in a display mode according to the (P S113).

[0068] The viewing status video is transmitted from the communication unit 31 to the distribution device 10 under the control of the communication control unit 34. The distribution device 10 receives the transmitted viewing status video (Step S114). The viewing status video is acquired by the communication control unit 22 and output by the output control unit 23. The result is displayed on the display unit 19 (step S116).

[0069] The response information acquisition unit 24 acquires a response from the performer to the viewing situation image displayed on the display unit 19. If so, the communication control unit 22 acquires the response information of the performer (step S117). The response information acquired by the response information acquisition unit 24 is transmitted from the communication unit 13 to the server device 30. (Step S118). The server device 30 receives the response information transmitted from the distribution device 10. (step S119).

[0070] The server device 30 changes the display mode of at least a part of the video to be distributed based on the response information. The display unit of the receiving device 50 of the audience to whom the performer responds is In the area 59, the icon 74 is displayed in a changed display mode (step S121). The audience can visually recognize the change in the display state of the icon 74 and can sense the response of the performer. In the flowchart shown in FIG. 12, the distribution device 10, the server device After each operation in the device 30 and the receiving device 50 is completed, the process is ended. Repeats a series of operations until an operation is received, a specified time has elapsed, or a specified number of times (loop) has elapsed. It is preferable to perform the process repeatedly (loop).

[0071] As described above, the distribution system 100 according to the present embodiment distributes the performance video of a performer. A distribution device 10 receives the demonstration video 73 and transmits the obtained demonstration video 73; A demonstration video 73 is received, and a distribution video 70 is generated based on the received demonstration video 73. A server device 30 that transmits the generated distribution video 70, and a A receiving device 50 that receives the distribution video 70 and displays the received distribution video 70 on a display unit 59. The receiving device 50 receives a viewer video 81 that is a video of a viewer viewing the video 70 for distribution. The server device 30 receives the viewer image 81 and transmits the received viewer image 81. The viewer video 81 is received, and a display according to the viewing state of the viewer viewing the distribution video 70 is displayed. A viewing status image 80 is generated in which a viewer image 81 is displayed or not displayed in a display mode. The distribution device 10 transmits the viewing status video 80 transmitted from the server device 30. The viewing status image 80 is received, and the received viewing status image 80 is displayed on the display unit 19.

[0072] According to this configuration, the viewer video 81 is acquired by the receiving device 50, and the server device 30 The viewing status video 80 is generated in the distribution device 10, and the viewing status video 80 is displayed on the display unit 19, the performer can see the viewing situation image 80 displayed on the display unit 19. This makes it possible to easily grasp the viewing status of the viewer.

[0073] In the distribution system 100 according to the present embodiment, the distribution device 10 displays on the display unit 19 Response information is obtained when the performer responds to the received viewer image 81, and the obtained response information is The server device 30 receives the response information transmitted from the distribution device 10. Based on the received response information, the display mode of at least a part of the video for distribution 70 is changed. According to the study, viewers can learn about the changes in the display mode of the distribution video 70 by observing the changes in the display mode of the distribution video 70. This allows the performer to communicate with the audience more easily. This can be easily done.

[0074] In the distribution system 100 according to the present embodiment, the distribution device 10 includes a performer and a display unit 19. The viewer's video 81 is displayed in a display mode according to the positional relationship with the performer. This makes it easier for the viewer to recognize the viewer image 81.

[0075] In the distribution system 100 according to this embodiment, the server device 30 displays a demonstration video 73. A display area 71 for displaying the live performance image 73 and an address of a viewer who views the live performance image 73 corresponding to the display area 71 are shown. A distribution image 70 including an icon arrangement area 72 in which an icon 74 can be arranged is generated, and a status information The information includes the positional relationship between the display area 71 and the icon 74. The viewing status image 80 is generated in various display modes according to the positional relationship between the display 1 and the icon 74. It is possible.

[0076] In the distribution system 100 according to the present embodiment, the status information is The server device 30 includes the viewer's enthusiasm, and the viewer video 8 of the viewer whose enthusiasm is below the standard. The viewing status video 80 is generated so that the distribution video 71 is not displayed. The viewing status of viewers who are highly positive about the program can be selectively displayed on the display unit 19. do.

[0077] The technical scope of the present invention is not limited to the above-described embodiment, and any other modifications and variations are possible without departing from the spirit of the present invention. For example, in the above embodiment, the distribution device 10 may Although the above description has been given with reference to an example in which the environment in which the event is held is one in which there are no spectators, the present invention is not limited to this. FIG. 13 is a schematic diagram showing another example of the distribution device 10 and devices connected to the distribution device 10. As shown in FIG. 13, a configuration in which audience seats 26 are arranged relative to a performance space 25. In this case, the display units 19A, 19B, and 19C can be viewed from the demonstration space 25. With this configuration, the performer can hear the audience in the auditorium 26. It is possible to grasp both the reaction of the audience at the audience seating 26 and the reaction of the audience at the receiving device 50. The audience and the viewers of the receiving device 50 can share their reactions to the performance by the performer. Also, the demonstration video 73 may be a plurality of videos. For example, By making the size of the icon 74 larger than the size of the demonstration video 73 and the icon 74, the viewer can The user can move the icon 74, approach the demonstration video 73 as needed, change the display range, etc. For example, a demonstration video73 can be posted on multiple websites. , so that viewers can watch multiple demonstration videos by moving between multiple websites. By arranging multiple demonstration videos 73, multiple performers can be simultaneously displayed on multiple stages. Events, so-called festivals, exhibitions, or conferences with multiple lectures, sometimes with demonstrations It is also possible to hold events such as poster sessions at the same time. In all the embodiments described above, the functions (operations) are implemented by the distribution device 10, the server device 30, or as part of the receiving device 50, but these are merely examples. As a whole system, each of the distribution device 10, the server device 30, and the receiving device 50 It is sufficient that the function (operation) is performed, and there is no restriction on the entity that performs the function (operation). The distribution system 100 includes only the functions (operations) of the distribution device 10 and the server device 30. In addition, the configuration may not include each function (operation) of the receiving device 50. [Explanation of symbols]

[0078] P1...performer, NW...network, 10...distribution device, 11,51...input unit, 12,5 2...output unit, 13, 31, 53...communication unit, 14, 32, 54...control unit, 15, 33, 55 ...Memory unit, 16, 56...Photographing unit, 17, 57...Audio input unit, 18, 58...Operation unit, 19, 19A, 19B, 19C, 59... display unit, 20, 60... audio output unit, 21, 61... input information information acquisition unit, 22, 34, 62 ... communication control unit, 23, 63 ... output control unit, 24 ... response information acquisition unit 25: performance space; 26: audience seats; 30: server device; 35: video generation unit for distribution , 36 ... viewing status image generating unit, 37 ... volume setting unit, 50 ... receiving device, 70 ... distribution image, 71: display area, 72: icon arrangement area, 72a: vicinity area, 73: demonstration video, 74: Icon, 75...Background image, 80...Viewing status image, 81...Viewer image, 100...Distribution system TEAM

Claims

1. A distribution system comprising: A function of acquiring a viewer image showing a viewer who is viewing the video; a function of generating a distribution video including a display area for displaying the video, and an icon arrangement area corresponding to the display area and in which a viewer who watches the video can arrange his / her own icon that can be moved; A function to display the viewing situation video being viewed by the performer on a display that the performer can see, Equipped with Whether or not to display the icon in the viewing status video is controlled based on a positional relationship between the display area and the icon in the distributed video. Delivery system.

2. The distribution system according to claim 1 , wherein the icon corresponds to any one of an image, a video of the viewer, and an avatar.

3. The distribution system according to claim 1 , wherein the distributed video is synthesized with a viewer image showing a viewer watching the video in an icon arrangement area in place of or in addition to the icon based on a set public range.

Images