Display video selection device, display video selection method, and display video selection program

Abstract

According to an embodiment, this displayed video selecting device, which selects, from among a plurality of remote audience videos captured at one or more event venues and transmitted via a communication network, a predetermined number of remote audience videos that are to be displayed on video display devices located at the event venues, comprises a video state determining unit and a video output content determining unit. The video state determining unit analyzes the remote audience videos and evaluates the excitement degrees of remote audience appearing in the remote audience videos. The video output content determining unit selects a predetermined number of remote audience videos in decreasing order of the values of the excitement degrees.

Description

[Technical Field]

[0001] One aspect of the present invention relates to a display video selection device, a display video selection method, and a display video selection program. [Background technology]

[0002] In recent years, video and audio playback devices have come into use to digitize video and audio filmed and recorded at a certain location, transmit them in real time to a remote location via communication lines such as an IP (Internet Protocol) network, and play them back at the remote location. For example, event video distribution and public viewing, in which video and audio from events taking place at sporting venues, concert halls, fashion show venues, and other event venues are transmitted in real time to remote locations, are becoming increasingly common.

[0003] Such video and audio transmission is not limited to one-to-one, one-way transmission; it can also be bidirectional. In this bidirectional transmission, for example, video and audio are transmitted from an event venue where a concert is being held to one or more remote venues, where they are output from monitors or large-screen display devices and speakers. Then, video and audio of the audience at those remote venues (hereinafter referred to as "remote audience") enjoying the event and audio, such as cheers, are filmed and recorded, and transmitted to the event venue, where they are aggregated and output from large displays and speakers. This bidirectional video and audio transmission allows performers, athletes, and audience members at the event venue to experience a sense of presence and unity, as if they were in the same space (event venue) and experiencing the same thing, despite the remote audience being physically distant from them.

[0004] Non-patent document 1 proposes that the faster the penlights move in video of multiple remote spectators (hereinafter referred to as remote spectator video), the higher the likelihood that the remote spectators are excited, and that cheering sounds be played according to the excitement. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Takayuki Kurozumi, Kaoru Hasegawa, Eiichiro Matsumoto, Toshihiko Eura, Shinji Fukatsu, "Technology for Reproducing Harmony Between Real Venue and Remote Audiences," NTT Technical Journal, September 2022 issue, September 1, 2022, Internet<URL:https: / / journal.ntt.co.jp / wp-content / uploads / 2022 / 09 / nttjnl5004_20220901.pdf> Summary of the Invention [Problem to be solved by the invention]

[0006] Meanwhile, in terms of display, the large display at the event venue is designed to simultaneously display multiple remote audience videos, for example, in a 5x5 tile arrangement. Therefore, if the number of remote audience videos transmitted from the remote venue exceeds the number that the large display can simultaneously display, it becomes necessary to select the remote audience videos to display according to this limited number. The more excited the remote audience videos displayed on the large display are, the more excited the audience at the event venue will be.

[0007] To create excitement using remote audience footage, it's best if all of the remote audience footage being displayed is enthusiastic, but it's not guaranteed that all of the remote audience members will move in the right way at the right time to appear excited. Furthermore, problems with the communication environment can sometimes cause disruptions to the footage. To prevent inappropriate footage from being displayed at the event venue, a human being visually identifies the footage and manually switches the displayed footage, but there is a limit to the number of images a human can process.

[0008] It is possible to use the method disclosed in Non-Patent Document 1 when selecting remote audience footage, but as mentioned above, there are several points to consider in addition to the movement of the penlights when determining whether the footage is appropriate for creating excitement.

[0009] This invention was made in light of the above circumstances, and its purpose is to provide a technology that makes it possible to select the remote audience video that is most exciting from among multiple remote audience videos. [Means for solving the problem]

[0010] In one embodiment of the present invention, a display video selection device selects a predetermined number of remote spectator videos to be displayed on a video display device installed at an event venue from among multiple remote spectator videos captured at one or more remote venues and transmitted via a communications network, and includes a video state determination unit and a video output content determination unit. The video state determination unit analyzes the remote spectator videos and quantifies the excitement levels of the remote spectators shown in the remote spectator videos. The video output content determination unit selects the predetermined number of remote spectator videos in descending order of excitement level. The individual display positions on the display array of the video display device correspond to the selection order of the remote spectator videos selected in descending order of the excitement level. The video output content determination unit selects the remote spectator videos at regular time intervals, and determines the display position of each selected remote spectator video on the display array of the video display device based on the selection order of the selected remote spectator videos. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to select a remote spectator video that is more exciting from among a plurality of remote spectator videos. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of each electronic device included in an event video display system to which a display video selection device according to a first embodiment of the present invention is applied. [Figure 2] FIG. 2 is a block diagram showing an example of the software configuration of a server device serving as a display video selection device according to the first embodiment. [Figure 3]FIG. 3 is a schematic diagram showing an example of a display arrangement of remote audience videos. [Figure 4] FIG. 4 is a flowchart showing the processing procedure and processing contents of the excitement level calculation processing in the server device. [Figure 5] FIG. 5 is a flowchart showing the processing procedure and processing contents of the remote audience video output processing in the server device. [Figure 6] FIG. 6 is a flowchart showing the processing procedure and processing contents of a remote audience video output process in a server device serving as a display video selection device according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, several embodiments of the present invention will be described with reference to the drawings.

[0014] [First embodiment] (Configuration example) FIG. 1 is a block diagram showing an example of the hardware configuration of each electronic device included in an event video display system S to which a display video selection device according to a first embodiment of the present invention is applied.

[0015] The event video display system S includes a plurality of electronic devices included in the event venue I and one or more (n in FIG. 1) remote venues R1 to R2. n The electronic devices at the event venue I and the remote venues R1 to Rn can communicate with each other via an IP network NW.

[0016] The event venue I includes a server device 1, an input / output device 101, a video shooting device 102, a sound recording device 103, and a remote audience video display device 104.

[0017] The server device 1 is an electronic device that controls each electronic device included in the event venue I. The server device 1 is an example of a display video selection device according to the first embodiment of the present invention.

[0018] The input / output device 101 is a user interface for an operator of the server device 1, and includes input devices such as a keyboard, a pointing device such as a mouse, and a display device such as a monitor. The operator of the server device 1 can be an event manager, a video director, a switcher who receives instructions from them, or the like.

[0019] The video shooting device 102 is a device including a camera that shoots video of the event venue I. The video shooting device 102 may include multiple cameras.

[0020] The sound recording device 103 is a device including a microphone that records sounds from the event venue I. The sound recording device 103 may include multiple microphones.

[0021] Remote spectator video display device 104 is a device including a large display that plays back and displays a predetermined number of remote spectator videos transmitted from each of remote venues R1 to Rn to event venue I. The display may be, for example, a liquid crystal display. Remote spectator video display device 104 is an example of a video display device. For example, at a concert, remote spectator video display device 104 may be installed behind performers on a stage where performers such as musicians and singers stand. Furthermore, at sports stadiums, remote spectator video display device 104 is often installed in a position away from the athletes.

[0022] An example of the configuration of the server device 1 at the event venue I will be described. The server device 1 includes a control unit 11, a program storage unit 12, a data storage unit 13, a communication interface 14, and an input / output interface 15. The elements included in the server device 1 are connected to each other via a bus.

[0023] The control unit 11 corresponds to the central part of the server device 1. The control unit 11 includes a processor such as a central processing unit (CPU). The processor may be multi-core / multi-threaded and can execute multiple processes in parallel. The control unit 11 includes a ROM (Read Only Memory) as a non-volatile memory area. The control unit 11 includes a RAM (Random Access Memory) as a volatile memory area. The processor loads a program stored in the ROM or the program storage unit 12 into the RAM. The processor executes the program loaded into the RAM, and the control unit 11 realizes each functional unit described below. The control unit 11 constitutes a computer.

[0024] The program storage unit 12 is configured with a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD) that can be written to and read from as needed as a storage medium. The program storage unit 12 stores programs necessary for executing various control processes. For example, the program storage unit 12 stores programs that cause the server device 1 to execute processes by each functional unit (described later) realized by the control unit 11. The program storage unit 12 is an example of storage.

[0025] The data storage unit 13 is configured with a nonvolatile memory that can be written to and read from as needed, such as an HDD or an SSD, as a storage medium. The data storage unit 13 is an example of a storage or a storage unit.

[0026] The communication interface 14 includes various interfaces that connect the server device 1 to other electronic devices so that they can communicate with each other, using a communication protocol defined by the IP network NW.

[0027] The input / output interface 15 is an interface that enables communication between the server device 1 and each of the input / output device 101, the video recording device 102, the audio recording device 103, and the remote audience video display device 104. The input / output interface 15 may be provided with an interface for wired communication or an interface for wireless communication.

[0028] The hardware configuration of the server device 1 is not limited to the above configuration. The server device 1 allows the above components to be omitted or changed as appropriate, and new components to be added.

[0029] The remote venue R1 is equipped with a remote terminal 2. The remote venue R1 may be a private home or a large facility such as a public viewing venue. If the remote venue R1 is a private home, the remote terminal 2 may be an information processing device such as a personal computer, a television with network connection functionality, or the like. If the remote venue R1 is a public viewing venue or the like, the remote terminal 2 may be a computer similar to the server device 1. The remote terminal 2 is equipped with a monitor 21, a speaker 22, and a camera 23.

[0030] The monitor 21 plays back and displays the video transmitted from the event venue I to the remote venue R1.

[0031] The speaker 22 reproduces and outputs the audio transmitted from the event venue I to the remote venue R1.

[0032] Camera 23 is a camera that captures images of remote spectators at remote venue R1 and generates remote spectator videos. Note that the number of cameras 23 at remote venue R1 is not limited to one, and there may be multiple cameras. In this case, the multiple cameras 23 are installed so as to capture images of different event spectators.

[0033] Remote Venue R2~Remote Venue R nThe hardware configuration of the electronic devices included in each of the above-mentioned remote venues R1 to R2 is the same as that of the above-mentioned remote venue R1, so the description thereof will be omitted. n When there is no need to distinguish between the two, we will simply refer to it as remote venue R.

[0034] FIG. 2 is a block diagram showing an example of the software configuration of server device 1 as a display video selection device according to the first embodiment. Server device 1 includes, as functional units, an event video distribution unit 111, a remote spectator video reception unit 112, a hidden spectator setting unit 113, a remote spectator video status determination unit 114, a remote spectator video output content determination unit 115, and a remote spectator video output unit 116. Each functional unit is realized by execution of a program stored in program storage unit 12 by control unit 11. It can also be said that each functional unit is included in control unit 11 or a processor. Each functional unit can be interchangeably referred to as control unit 11 or a processor. Each functional unit may also be realized in a variety of other forms, including integrated circuits such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit). Furthermore, the server device 1 includes a learned data storage unit 131, a display position designation data storage unit 132, a remote spectator video image storage unit 133, a non-displayed spectator video image storage unit 134, an excitement level storage unit 135, and an output content storage unit 136. The learned data storage unit 131, the display position designation data storage unit 132, the remote spectator video image storage unit 133, the non-displayed spectator video image storage unit 134, an excitement level storage unit 135, and the output content storage unit 136 are realized by the data storage unit 13.

[0035] The learned data storage unit 131 stores learned data created in advance that describes the movements of the audience that are desired to be displayed at the event venue I, that is, what kind of movements are considered to be exciting movements. Examples of such movements that can be used include the movement vectors of the people appearing in the remote audience video, that is, the remote audience themselves, and the movement vectors of objects held by the remote audience, such as penlights that are appearing in the video.

[0036] The display position designation data storage unit 132 stores display position designation data that indicates the display rules for multiple remote spectator videos on the remote spectator video display device 104. FIG. 3 is a schematic diagram showing an example of the display arrangement of remote spectator videos to illustrate one example of this rule. In this example, 25 remote spectator videos are arranged in tiles on the display screen 104A of the remote spectator video display device 104. The display position designation data thus describes the rules for where the remote spectator videos should be displayed, based on the ranking of the remote spectator's level of excitement. The example in FIG. 3 shows a case where the videos are specified to be arranged in a raster scan direction, starting from the top left, in order of increasing excitement.

[0037] The event video distribution unit 111 transmits the video output from the video shooting device 102 and the audio output from the audio recording device 103 to each of one or more remote venues R via the communication interface 14 and the IP network NW.

[0038] The remote audience video receiving unit 112 acquires the remote audience video transmitted from each of the remote venues R via the IP network NW, which is received by the communication interface 14. The remote audience video receiving unit 112 stores the acquired remote audience video in the remote venues R1 to R2 stored in the remote audience video storage unit 133. n Alternatively, the remote audience video receiving unit 112 may store the acquired remote audience video in the respective video storage areas. nThe remote spectator video receiving unit 112 continues to receive the remote spectator video as long as it is necessary to display it on the remote spectator video display device 104 at the event venue I. Each remote spectator video is stored in the remote spectator video storage unit 133 for only a predetermined period of time, and may be constantly updated or may be configured to delete the oldest specified period of remote spectator video each time the specified period of remote spectator video is accumulated. Note that it is desirable for the number of remote spectator videos to be equal to or greater than the number required for display on the remote spectator video display device 104.

[0039] The non-display spectator setting unit 113 receives the designation of a specific remote spectator video by operating the input / output device 101 and sets the remote spectator video as a non-display spectator video to be excluded from display on the remote spectator video display device 104. The non-display spectator setting unit 113 stores non-display spectator information indicating which remote spectator video has been set as a non-display spectator video in the non-display spectator memory unit 134.

[0040] The remote spectator video state determination unit 114 uses the learned data stored in the learned data storage unit 131 to analyze each remote spectator video stored in the remote spectator video storage unit 133, thereby quantifying the excitement level of the remote spectators shown in the remote spectator video. In addition to quantifying the movement of the penlights held by the remote spectators, the remote spectator video state determination unit 114 also quantifies states that are clearly inappropriate for display as a low excitement level. Examples of states that are clearly inappropriate for display include when the remote spectator is looking down (their eyes or head are downward), when the remote spectator is away (no person is detected in the remote spectator video), when there is a disturbance in the remote spectator video, or when the remote spectator video is not displayed (interrupted). The remote spectator video state determination unit 114 stores the quantified excitement level for each remote spectator video in the excitement level storage unit 135.

[0041] The remote spectator video output content determination unit 115 selects remote spectator video to display on the remote spectator video display device 104 based on the excitement level ranking stored in the excitement level storage unit 135. At this time, the remote spectator video output content determination unit 115 excludes remote spectator video from selection, even if the remote spectator video has a high excitement level ranking, based on the non-display spectator information stored in the non-display spectator storage unit 134. The remote spectator video output content determination unit 115 further determines the placement position of the selected remote spectator video on the display screen 104A of the remote spectator video display device 104 according to its excitement level ranking, in accordance with the display designation position data stored in the display position designation data storage unit 132. That is, the remote spectator video output content determination unit 115 determines which remote spectator video should be displayed at which position on the display screen 104A of the remote spectator video display device 104. The remote spectator video output content determination unit 115 stores output content data in the output content storage unit 136, indicating the correspondence between the determined placement position on the display screen 104A and the remote spectator video. The output content data may be, for example, metadata containing pairs of an identifier indicating the output position of the remote audience video and an identifier for each remote audience video, the number of pairs corresponding to the number of output positions.

[0042] The remote spectator video output unit 116 reads the relevant remote spectator videos from the remote spectator video storage unit 133 in accordance with the output content data stored in the output content storage unit 136, and creates an output video by arranging the remote spectator videos on one screen. The remote spectator video output unit 116 displays the created output video on the remote spectator video display device 104 via the input / output interface 15.

[0043] (Example of operation) An example of the operation of the server device 1 as a display video selection device according to the first embodiment of the present invention will be described below. Note that the operation related to the distribution of event videos is the same as in the conventional case, and therefore a description thereof will be omitted.

[0044] 4 is a flowchart showing the processing procedure and processing content of the excitement level calculation processing in the server device 1. The server device 1 repeatedly executes the processing shown in this flowchart while the remote audience video display device 104 is displaying the remote audience video at the event venue I.

[0045] First, the remote spectator video state determination unit 114 reads out one of the multiple remote spectator videos stored in the remote spectator video storage unit 133 as a processing target (step S101).

[0046] Then, remote spectator video state determination unit 114 determines whether or not there is any disturbance or interruption in the remote spectator video to be processed that has been read out (step S102).

[0047] If it is determined that there is no disturbance or interruption in the remote spectator video to be processed, the remote spectator video state determination unit 114 further determines whether the remote spectator video to be processed includes a person (step S103). This determination can be made using a known person detection algorithm or the like.

[0048] If it is determined that the remote spectator video to be processed includes a person, remote spectator video state determination unit 114 further determines whether the person included in the remote spectator video to be processed is looking away (step S104). This determination can be made using a known eye detection algorithm or the like.

[0049] If it is determined that the person in the remote spectator video to be processed is not looking away, the remote spectator video state determination unit 114 detects the movement of the person or object in the remote spectator video to be processed (step S105). This detection can be performed using a known motion vector detection algorithm or the like.

[0050] Furthermore, the remote spectator image state determination unit 114 calculates a numerical value x of the excitement level based on the detected movement using the learned data stored in the learned data storage unit 131 (step S106). For example, the remote spectator image state determination unit 114 determines, using the learned data, whether the detected movement is a movement to be used for calculating the excitement level, and if so, determines the magnitude of the motion vector of the detected movement as the excitement level.

[0051] Then, remote spectator image state determination unit 114 stores the calculated excitement level value x as the excitement level of the remote spectators in the remote spectator image being processed in excitement level storage unit 135 (step S107).

[0052] Thereafter, the remote spectator image status determination unit 114 determines whether there is any unprocessed remote spectator image in the remote spectator image storage unit 133, that is, whether all of the multiple stored remote spectator images have been processed (step S108).

[0053] If it is determined that there is unprocessed remote spectator video, remote spectator video state determination unit 114 proceeds to the processing of step S101 and repeats the above processing.

[0054] Furthermore, if it is determined in step S104 that a person included in the remote spectator video being processed is looking away, for example, with their eyes or head downward, remote spectator video state determination unit 114 sets the excitement level to a numerical value y (step S109). This excitement level numerical value y is a constant that is lower than the numerical value x calculated in step S106 (i.e., x>y). Then, remote spectator video state determination unit 114 proceeds to the processing of step S107, where it stores this excitement level numerical value y in excitement level storage unit 135 as the excitement level of the remote spectator in the remote spectator video being processed.

[0055] If it is determined in step S103 that the remote spectator video to be processed does not include any people, i.e., that the remote spectator has left their seats, remote spectator video state determination unit 114 sets the excitement level to z (step S110). This excitement level value z is a constant that is lower than the value y determined in step S109 (i.e., x>y>z). Remote spectator video state determination unit 114 then proceeds to the processing of step S107, where it stores this excitement level value z in excitement level storage unit 135 as the excitement level of the remote spectator in the remote spectator video to be processed.

[0056] Furthermore, if it is determined in step S102 that there is a disturbance or interruption in the remote spectator video being processed, the remote spectator video state determination unit 114 sets the excitement level value to "0" (step S111). Note that the value z calculated in step S110 is higher than "0" (i.e., x>y>z>0). The remote spectator video state determination unit 114 then proceeds to the processing of step S107, where it stores this excitement level value "0" in the excitement level storage unit 135 as the excitement level of the remote spectators in the remote spectator video being processed.

[0057] Once the above processing has been performed on all of the multiple remote spectator images stored in the remote spectator image storage unit 133, it is determined in step S108 above that there are no unprocessed remote spectator images, and the remote spectator image state determination unit 114 terminates the excitement level calculation processing shown in this flowchart.

[0058] As described above, the server device 1 repeatedly executes the process shown in this flowchart while the remote spectator video display device 104 is displaying the remote spectator video at the event venue I. Therefore, the remote spectator video state determination unit 114 starts the process again from step S101.

[0059] 5 is a flowchart showing the processing procedure and processing content of the remote audience video output processing in the server device. When the server device 1 receives an operation on the input / output device 101 instructing the remote audience video display device 104 to start displaying the remote audience video, the server device 1 starts the processing shown in this flowchart.

[0060] First, the non-display audience setting unit 113 determines whether or not a non-display audience designation specifying that a specific remote audience video be hidden has been received from the input / output device 101 (step S201).

[0061] If it is determined that no non-display spectators have been designated, the remote spectator video output content determination unit 115 determines whether a certain period of time has elapsed (step S202). If it is determined that the certain period of time has not yet elapsed, the server device 1 proceeds to the process of step S201.

[0062] On the other hand, if it is determined that a certain amount of time has passed, the remote audience video output content determination unit 115 ranks the remote audience videos stored in the excitement level memory unit 135 in descending order of the remote audience excitement level value of each remote audience video (step S203).

[0063] Then, remote spectator video output content determination unit 115 performs a display position determination to determine which remote spectator video should be assigned to which position in order of priority, based on the display position designation data stored in display position designation data storage unit 132 (step S204). For example, if there are 30 remote spectator videos, in the example shown in Figure 3, remote spectator video output content determination unit 115 selects the 25 remote spectator videos with the highest priority as the remote spectator videos to be displayed. Then, remote spectator video output content determination unit 115 determines the display position for each of the selected remote spectator videos, based on the display arrangement on display screen 104A of remote spectator video display device 104 shown in Figure 3.

[0064] Here, the remote spectator video output content determination unit 115 determines whether or not the remote spectator video for which the display position has been determined includes remote spectator video designated as hidden, based on the hidden spectator information stored in the hidden spectator memory unit 134 (step S205).

[0065] If it is determined that the remote spectator video output content determination unit 115 does not include remote spectator video designated as hidden, it stores the display position determination result from step S204 as output content data in the output content storage unit 136 (step S206). That is, based on the display position determination result, the remote spectator video output content determination unit 115 stores in the output content storage unit 136 data indicating which remote spectator video should be output to which position on the display screen 104A.

[0066] The remote spectator video output unit 116 causes the remote spectator video display device 104 to display a plurality of remote spectator videos in accordance with the output content data stored in the output content storage unit 136 (step S207).

[0067] Thereafter, the server device 1 determines whether the event has ended (step S208). If it determines that the event has not yet ended, the server device 1 proceeds to the processing of step S201. In this way, the remote spectator video displayed on the remote spectator video display device 104 is updated every time a certain period of time passes.

[0068] If it is determined in step S201 that a spectator has been designated as not to be hidden, the non-display spectator setting unit 113 stores non-display designated spectator information indicating the remote spectator video designated as not to be hidden in the non-display spectator storage unit 134 (step S209). Of course, the number of remote spectator videos designated as not to be hidden is not limited to one. Then, the server device 1 proceeds to the processing of step S201.

[0069] When one or more non-display spectators are designated in this manner, it is determined in step S205 that a non-display designation is included. In this case, remote spectator video output content determination unit 115 further checks the excitement level values ​​of each remote spectator video indicated by the non-display spectator setting information stored in excitement level storage unit 135, and determines whether any remote spectator video has an excitement level equal to or greater than a threshold (step S210). This threshold value may be set to a value higher than the aforementioned excitement level value y, i.e., a value within the range of value x, thereby making it possible to determine whether the conditions are appropriate for displaying the remote spectator video.

[0070] If it is determined that there are no remote spectator videos with an excitement level value equal to or greater than the threshold, the remote spectator video output content determination unit 115 reflects the non-display designation in the display position determination result from step S204 (step S211). That is, the remote spectator video output content determination unit 115 excludes the remote spectator videos designated as non-display from the remote spectator videos to be displayed, and selects the remote spectator video with the highest ranking from the remaining remote spectator videos to determine the display position. The remote spectator video output content determination unit 115 then proceeds to the processing of step S206, where the display position determination result reflecting the non-display designation is stored in the output content storage unit 136 as output content data. In this way, the non-display selection manually designated by the operator of the server device 1 since the previous processing performed a certain time ago can be reflected in the display position determination result.

[0071] Furthermore, if it is determined in step S210 that there is remote spectator video with an excitement level value equal to or greater than the threshold, the remote spectator video output content determination unit 115 cancels the hide designation for that remote spectator video (step S212). That is, the remote spectator video output content determination unit 115 creates new hide-designated spectator information by excluding remote spectator video with an excitement level value equal to or greater than the threshold from the hide-designated spectator information stored in the hide-designated spectator storage unit 134, and stores this information in the hide-designated spectator storage unit 134. The remote spectator video output content determination unit 115 then proceeds to the processing of step S211 and generates a display position determination result that reflects the remote spectator video designated as hide-designated other than the remote spectator video equal to or greater than the threshold.

[0072] If it is determined in step S208 that the event has ended, the server device 1 ends the processing shown in this flowchart.

[0073] (effect) As described above, in the first embodiment, when server device 1 selects a predetermined number of remote spectator videos to be displayed on remote spectator video display device 104, which is a video display device installed at event venue I, from among multiple remote spectator videos taken at one or more remote venues R and transmitted via IP network NW, which is a communications network, remote spectator video state determination unit 114 analyzes the remote spectator videos and quantifies the levels of excitement of the remote spectators shown in the remote spectator videos, and remote spectator video output content determination unit 115 selects the predetermined number of remote spectator videos in descending order of excitement level. In this way, the levels of excitement of the remote spectators shown in the remote spectator videos are quantified and the predetermined number of remote spectator videos to be displayed in descending order of these numerical values ​​are selected, making it possible to select remote spectator videos with a higher level of excitement from among multiple remote spectator videos.

[0074] The remote spectator video output content determination unit 115 selects remote spectator videos at regular intervals and determines the positions of the selected remote spectator videos in the display array on the display screen 104A of the remote spectator video display device 104 according to rules determined by the order in which the remote spectator videos were selected, which are stored as display designation position data in the display position designation data storage unit 132. Thus, at regular intervals, it is possible to select the remote spectator video that is appropriate for display at that time.

[0075] Furthermore, in the first embodiment, the server device 1 includes a hidden spectator setting unit 113 that receives input specifying a remote spectator video to be hidden from among the selected remote spectator videos and deselects the specified remote spectator video. This allows the operator of the server device 1 to determine whether the video is appropriate for creating excitement and manually specify the remote spectator video to be displayed.

[0076] Furthermore, the remote spectator video state determination unit 114 quantifies the excitement level based on at least one of the following: the movement vector of the remote spectator themselves or an object held by the remote spectator, the orientation of the remote spectator, the presence or absence of a remote spectator, and a disturbance or interruption of the remote spectator video. Therefore, by quantifying the excitement level based on various aspects other than the movement of the object held by the remote spectator, such as the movement of a penlight, more accurate quantification is possible.

[0077] [Second embodiment] In the first embodiment described above, the remote spectator video displayed on the remote spectator video display device 104 is updated every time a certain period of time has elapsed, but the update may be performed only when a new remote spectator video is to be replaced.

[0078] Hereinafter, the same configurations and processes as those in the first embodiment will be denoted by the same reference numerals as those in the first embodiment, and the description thereof will be omitted, and only the portions different from the first embodiment will be described.

[0079] FIG. 6 is a flowchart showing the processing procedure and processing contents of a remote audience video output process in a server device serving as a display video selection device according to the second embodiment of the present invention.

[0080] In the second embodiment, if it is determined in step S205 that the display does not include a hide command, and following step S211, the remote spectator video output content determination unit 115 determines whether a change in the ranking of the remote spectator videos between the previous processing and the current processing has occurred (step S220). For example, the remote spectator video output content determination unit 115 can determine whether a change in the selected remote spectator video has occurred by comparing the display position determination result performed in step S204 a certain time ago with the display position determination result performed in step S204 this time, based on the output content data stored in the output content storage unit 136. In other words, the remote spectator video output content determination unit 115 determines whether a remote spectator video has been removed from the selection to be displayed and has been replaced by another remote spectator video.

[0081] If it is determined that there has been no change, in the second embodiment, the server device 1 proceeds to step S207 and causes the remote spectator video display device 104 to display multiple remote spectator videos in accordance with the output content data stored in the output content storage unit 136. That is, even if the order of the remote spectator videos has changed since a certain period of time ago, the arrangement of the remote spectator videos on the display screen 104A is maintained. This prevents the display position of the remote spectator videos of the same remote spectator from changing.

[0082] If step S220 determines that a change has occurred, the remote spectator video output content determination unit 115 assigns display positions (step S221). Specifically, the remote spectator video output content determination unit 115 hides the remote spectator video of the remote spectator whose rank falls below the desired number of people to be displayed, and specifies that the video of the remote spectator whose rank now exceeds the desired number of people to be displayed be displayed in the vacant position. Specifically, assuming 25 images are to be displayed as in the example of FIG. 3, if any remote spectator video that was within the top 25 in terms of excitement level in the previous (n-1)th processing run has fallen below 25th place in the current (nth) run, the remote spectator video output content determination unit 115 replaces the remote spectator video that was within the top 25 in terms of excitement level in the nth run. For example, if the rankings in the n-1th and nth runs were 17th to 26th (remote spectator video A) and 27th to 23rd (remote spectator video B), the remote spectator video B would be assigned to the 17th position where remote spectator video A had been displayed. If there are multiple remote spectator videos to be swapped, the location with the highest n-1th order will be assigned in descending order of nth order. Remote spectator video output content determination unit 115 then proceeds to the process of step S206, and the display position determination result reflecting this display position swap is saved in output content storage unit 136 as output content data.

[0083] (effect) As described above, in the server device 1 of the second embodiment, the remote spectator video output content determination unit 115 does not change the display arrangement on the video display device if the remote spectator video continues to be selected, even if the excitement level of the remote spectator video has changed from the excitement level value from a certain time ago. This prevents the display position of the remote spectator video of the same remote spectator from changing.

[0084] Furthermore, when the excitement level of a remote spectator video changes from the excitement level value from a certain time ago and that remote spectator video is no longer selected, remote spectator video output content determination unit 115 places the newly selected remote spectator video in the position on the display array where the no longer selected remote spectator video was placed. Thus, only when a replacement has occurred, the remote spectator video can be replaced and displayed only in the display position where the replacement occurred.

[0085] [Other embodiments] The flow of each process described with reference to the flowchart is not limited to the described processing procedure. The order of some steps may be changed, some steps may be performed simultaneously in parallel, or the processing content of some steps may be modified.

[0086] Furthermore, the server device 1 may be realized by one device as explained in the above example, or may be realized by a plurality of devices with distributed functions.

[0087] Furthermore, the information processing program according to each embodiment and the methods described in other embodiments can be stored as a processing program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by having the operation controlled by this software means. The recording medium referred to in this specification is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via an IP network NW.

[0088] In short, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0089] 1...Server device 2...Remote terminal 11...Control unit 12...Program memory section 13...Data storage unit 14...Communication interface 15...Input / output interface 21...Monitor 22...Speaker 23...Camera 101...input / output device 102...Video recording equipment 103...Audio recording device 104...Remote audience image display device 104A…Display screen 111...Event Video Distribution Department 112...Remote audience video receiving unit 113...Hidden audience setting section 114...Remote audience video status judgment unit 115...Remote audience video output content judgment unit 116...Remote audience video output unit 131... Learned data storage unit 132...Display position designation data storage unit 133...Remote audience video storage unit 134...Hidden Audience Memory Section 135...Excitement memory section 136...Output content storage unit I…Event venue NW...IP network R,R1,R2,R n …Remote venue S...Event video display system

Claims

1. A display video selection device that selects a predetermined number of remote spectator videos to be displayed on a video display device installed at an event venue from a plurality of remote spectator videos taken at one or more remote venues and transmitted via a communication network, a video state determination unit that analyzes the remote audience video and quantifies the level of excitement of the remote audience shown in the remote audience video; a video output content determination unit that selects the predetermined number of remote audience videos in descending order of the excitement level; Equipped with the individual display positions on the display array of the video display device are associated with the selection order of the remote spectator videos, which are selected in descending order of the excitement level; The video output content determination unit Selecting the remote audience video at regular intervals, a display image selection device that determines the display position on the display array of the image display device of each of the selected remote spectator images based on the selection order of the selected remote spectator images;

2. The display video selection device of claim 1, wherein the video output content determination unit does not change the display arrangement on the video display device if the remote audience video continues to be selected even if the excitement level value of the remote audience video changes from the excitement level value from the certain time ago.

3. The display video selection device of claim 2, wherein when the excitement level value of the remote spectator video changes from the excitement level value from a certain time ago and the remote spectator video is no longer selected, the video output content determination unit places the newly selected remote spectator video in the position on the display array where the no longer selected remote spectator video was placed.

4. The display video selection device of claim 1 further comprises a hidden audience setting unit that receives a designation input of a remote audience video that should be hidden from among the selected remote audience videos and deselects the designated remote audience video.

5. The display video selection device of claim 1, wherein the video state determination unit quantifies the excitement level based on at least one of the movement vector of the remote spectator itself or an object held by the remote spectator, the orientation of the remote spectator, the presence or absence of the remote spectator, and distortion or interruption of the remote spectator video.

6. A display video selection method executed by a display video selection device having a processor and a memory, the display video selection device selecting a predetermined number of remote spectator videos to be displayed on a video display device installed at an event venue from a plurality of remote spectator videos taken at one or more remote venues and transmitted via a communication network, the method comprising: The processor analyzes the remote audience video transmitted via the communication network and stored in the memory, quantifies the degree of excitement of the remote audience shown in the remote audience video, and stores the quantified degree of excitement of the remote audience in the memory; the processor selects the predetermined number of the remote spectator videos in descending order of the excitement level values ​​stored in the memory; Including, the individual display positions on the display array of the video display device are associated with the selection order of the remote spectator videos, which are selected in descending order of the excitement level; The processor selecting the predetermined number of the remote spectator videos includes: selecting the remote audience video at regular intervals; determining the display positions of the selected remote spectator videos on the display array of the video display device according to the selection order of the selected remote spectator videos; A display image selection method including:

7. 6. A display video selection program that causes a computer to execute processing by each unit included in the display video selection device according to claim 1.

Images