Playback control method, control system, and program
The method improves remote event participation by selectively processing and playing user voices from multiple terminals, ensuring clear audio quality and engagement.
Patent Information
- Application Number
- JP2020155484
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-09-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-09-16
AI Technical Summary
Existing systems for remote participation in events, such as sporting or music events, mix voices from multiple users, making it difficult for viewers to clearly hear individual user voices.
A method and system that receives input data from multiple terminal devices in parallel, selects and plays a subset of this data in real-time, applying acoustic processing to enhance clarity, and optionally mixes it with event audio.
Enhances the clarity of individual user voices, allowing remote participants to better engage with the event atmosphere, while reducing processing load and maintaining clear sound quality.
Smart Images

Figure 0007593018000001 
Figure 0007593018000002 
Figure 0007593018000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to techniques for controlling sound reproduction. [Background technology]
[0002] Systems have been proposed for allowing many users in remote locations to participate in various events, such as sporting events, music events, etc. For example, Patent Document 1 discloses a technology for playing back audio input by users in remote locations to a terminal device on an output device installed at the location where the event is held (hereinafter referred to as the "event location"). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2015-507855 Summary of the Invention [Problem to be solved by the invention]
[0004] However, in the technology of Patent Document 1, the voices of many users in remote locations are mixed and played back by an output device at the event location. Therefore, there is a problem that the voices of each user cannot be clearly heard by the event spectators. In consideration of the above circumstances, one aspect of the present disclosure aims to make it easier to hear sounds corresponding to data received from multiple terminal devices at an event. [Means for solving the problem]
[0005] In order to solve the above problems, a playback control method according to one embodiment of the present disclosure receives multiple input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, selects one or more first input data that are part of the multiple input data, and plays a first sound corresponding to the one or more first input data in parallel with the progress of the event.
[0006] A control system according to one embodiment of the present disclosure includes a receiving processing unit that receives a plurality of input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, a selection processing unit that selects one or more first input data that are part of the plurality of input data, and a playback control unit that plays a first sound corresponding to the one or more first input data in parallel with the progress of the event.
[0007] A program according to one embodiment of the present disclosure causes a computer to function as a receiving processing unit that receives multiple input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, a selection processing unit that selects one or more first input data that are part of the multiple input data, and a playback control unit that plays a first sound corresponding to the one or more first input data in parallel with the progress of the event. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram illustrating a configuration of an information system in a first embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating a configuration of a playback device. [Diagram 3] FIG. 2 is a block diagram illustrating a configuration of a terminal device. [Figure 4] FIG. 2 is a block diagram illustrating a configuration of a control system. [Diagram 5] FIG. 2 is a block diagram illustrating a functional configuration of the control system. [Figure 6] 10 is a flowchart illustrating a specific procedure of a playback control process. [Figure 7] FIG. 11 is a block diagram illustrating a functional configuration of a control system according to a second embodiment. [Figure 8] 10 is a flowchart illustrating a specific procedure of a reproduction control process in the second embodiment. [Figure 9] FIG. 13 is an explanatory diagram regarding the operation of a playback control unit in the third embodiment. [Figure 10]FIG. 13 is a block diagram illustrating a configuration of a playback control unit in a fourth embodiment. [Figure 11] FIG. 13 is a block diagram illustrating a partial configuration of an information system according to a fifth embodiment. [Figure 12] 13 is a flowchart illustrating a specific procedure of a reproduction control process in the fifth embodiment. [Figure 13] FIG. 13 is a block diagram illustrating an example of a functional configuration of a control system in a sixth embodiment. [Figure 14] 23 is a flowchart illustrating a specific procedure of a reproduction control process in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] A: First embodiment 1 is a block diagram illustrating an example of the configuration of an information system 100 according to the first embodiment. The information system 100 is a computer system that provides services related to an event held in a facility 200 to a plurality of users Ua.
[0010] For example, various events such as a sporting event in which one or more contestants take part, or a music event in which one or more performers play or sing music, are held in the facility 200. The facility 200 in which the sporting event is held is, for example, a stadium or a gymnasium. The facility 200 in which the music event is held is, for example, an acoustic hall, a live music venue, or an outdoor stage.
[0011] A plurality of users Ub who use the facility 200 are present within the facility 200. The plurality of users Ub include, for example, contestants who will be taking part in a sporting event, performers who will be appearing at a music event, an administrator who will be managing and operating the event, and spectators who will be watching the event. Note that due to various circumstances, such as preventing the spread of infectious diseases, an event may be held without any spectators present within the facility 200.
[0012] Each of the multiple users Ua is located outside the facility 200. Specifically, the user Ua is located at a location remote from the facility 200 (for example, at home). However, the user Ua may be located within the facility 200. Each user Ua can use a playback device 21 and a terminal device 22. The terminal device 22 is, for example, a portable information terminal such as a smartphone or a tablet terminal. Note that a stationary personal computer may also be used as the terminal device 22.
[0013] The reproducing device 21 reproduces the content C representing the status of an event held in the facility 200. FIG. 2 is a block diagram illustrating the configuration of the reproducing device 21. The content C is composed of video data C1 representing video and audio data C2 representing sound. The reproducing device 21 includes a display device 31 that displays the video represented by the video data C1, and an audio output device 32 that reproduces the sound represented by the audio data C2. For example, a television receiver that reproduces a television program transmitted by airwaves as the content C is used as the reproducing device 21. Also, an information terminal such as a smartphone or a tablet terminal is used as the reproducing device 21. A large-scale video device (public viewing) that can be viewed by many users Ua may be used as the reproducing device 21. The content C may be a radio program composed only of sound.
[0014] 1, the information system 100 includes a recording system 11, a distribution system 12, a playback system 13, and a control system 14. The recording system 11 and the playback system 13 are installed in a facility 200. The recording system 11 communicates with the distribution system 12. The playback system 13 communicates with the control system 14. The distribution system 12 and the control system 14 may be configured as a single device.
[0015] The recording system 11 records a video of an event held in the facility 200. Specifically, the recording system 11 includes an imaging device that captures video of the event and a sound collection device that collects sound of the event (not shown). The recording system 11 generates a video composed of the video captured by the imaging device and the sound collected by the sound collection device.
[0016] The distribution system 12 distributes content C including the video recorded by the recording system 11 to each of the multiple playback devices 21. The distribution system 12 can also distribute the content C to each terminal device 22. Specifically, the distribution system 12 distributes the content C in real time in parallel with the progress of the event in the facility 200 (i.e., live distribution). Each playback device 21 receives the content C from the distribution system 12 via the communication network 300 and plays the content C in parallel with the progress of the event. The user Ua can grasp the status of the event in real time by watching the content C played by the playback device 21. As can be understood from the above explanation, the user Ua is a spectator of the event held in the facility 200 (specifically, a viewer of the content C).
[0017] The playback system 13 plays sounds within the facility 200. The playback system 13 includes, for example, a plurality of sound emitting devices (e.g., speaker devices) installed at different locations within the facility 200. A user Ub within the facility 200 can hear the sounds played by the playback system 13 within the facility 200. The control system 14 is a computer system that controls the playback of sounds by the playback system 13. The control system 14 can communicate with each of the plurality of terminal devices 22 via a communication network 300.
[0018] 3 is a block diagram illustrating the configuration of each terminal device 22. The terminal device 22 includes a control device 41, a storage device 42, a communication device 43, an operation device 44, a playback device 45, and a sound collection device 46. The terminal device 22 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.
[0019] The control device 41 is composed of one or more processors that control each element of the terminal device 22. For example, the control device 41 is composed of one or more types of processors such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).
[0020] The storage device 42 is one or more memories that store the programs executed by the control device 41 and various data used by the control device 41. The storage device 42 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 42 may be configured by combining a plurality of types of recording media. The communication device 43 communicates with the control system 14 via the communication network 300.
[0021] The operation device 44 is an input device that receives instructions from the user Ua. The operation device 44 is, for example, a plurality of operators operated by the user Ua, or a touch panel that detects contact by the user Ua.
[0022] The playback device 45 plays back videos. For example, the playback device 45 can play back the content C, similar to the playback device 21. For example, the playback device 45 includes a display device 31 that displays a video represented by video data C1, and a sound emitting device 32 that plays back a sound represented by sound data C2, similar to the playback device 21 illustrated in FIG. 2. Note that the playback device 45 may play back the content C that the communication device 43 receives from the distribution system 12. In other words, the playback device 21 may be omitted.
[0023] The sound collection device 46 is a microphone that collects the voice (hereinafter referred to as "input voice") uttered by the user Ua. Specifically, the sound collection device 46 generates an audio signal V that represents the waveform of the input voice. The user Ua produces the input voice to the sound collection device 46 at a desired time while watching the content C using the reproduction device 21 or the reproduction device 45. The input voice is, for example, voices cheering on the contestants or performers of the event, cheers for the progress of the event, or insults expressing dissatisfaction with the event. For convenience, an A / D converter that converts the audio signal V from analog to digital is omitted from the illustration. In the first embodiment, a configuration in which the sound collection device 46 is mounted on the terminal device 22 is illustrated, but the sound collection device 46, which is separate from the terminal device 22, may be connected to the terminal device 22 by wire or wirelessly.
[0024] The control device 41 generates input data D from the acoustic signal V generated by the sound collection device 46. The input data D in the first embodiment is acoustic data composed of a sample sequence representing the waveform of the input sound. That is, the input data D is data corresponding to an input by the user Ua (i.e., a voice input). The control device 41 generates the input data D in response to an operation by the user Ua on the operation device 44. For example, the input data D is generated from the acoustic signal V during a period during which the operation of the operation device 44 is continued, or during a period between two time points specified by the operation on the operation device 44. Note that the terminal device 22 may transmit the input data D in response to an action of the user Ua vibrating the terminal device 22, or the like.
[0025] The communication device 43 transmits the input data D generated by the control device 41 to the control system 14. As can be understood from the above description, the input data D is transmitted in parallel or sequentially from each of the multiple terminal devices 22 to the control system 14 in parallel with the progress of the event.
[0026] 4 is a block diagram illustrating an example of the configuration of the control system 14. The control system 14 includes a control device 51, a storage device 52, and a communication device 53. The control system 14 may be realized as a single device, or may be realized as a set of multiple devices configured separately from each other.
[0027] The control device 51 is composed of one or more processors that control each element of the control system 14. For example, the control device 51 is composed of one or more types of processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC.
[0028] The storage device 52 is one or more memories that store the programs executed by the control device 51 and various data used by the control device 51. The storage device 52 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 52 may be configured by combining multiple types of recording media.
[0029] The communication device 53 communicates with each of the multiple terminal devices 22 via the communication network 300. The communication device 53 receives input data D transmitted from each of the multiple terminal devices 22.
[0030] 5 is a block diagram illustrating an example of a functional configuration of the control system 14. The control device 51 executes a program stored in the storage device 52 to function as a plurality of functions (a reception processing unit 61, a selection processing unit 62, and a playback control unit 63).
[0031] The receiving processor 61 receives the input data D transmitted from each terminal device 22 by the communication device 53. That is, the receiving processor 61 receives a plurality of pieces of input data D representing voices input by users Ua of different terminal devices 22. The reception of the plurality of pieces of input data D by the receiving processor 61 is repeated at a predetermined cycle in parallel with the progress of the event.
[0032] The selection processing unit 62 selects N pieces of input data D (hereinafter referred to as "selected data D1") from the multiple pieces of input data D received by the reception processing unit 61. The number N of selected data D1 in the first embodiment is a natural number equal to or greater than 2. The number N is set to, for example, a fixed value specified in advance. The N pieces of selected data D1 are a portion of the multiple pieces of input data D. That is, the selection processing unit 62 selects a portion of the multiple pieces of input data D as the selected data D1. The selection processing unit 62 in the first embodiment randomly selects N pieces of selected data D1 from the multiple pieces of input data D. According to the above configuration, the processing load of the selection processing unit 62 selecting the N pieces of selected data D1 is reduced. The selection of the N pieces of selected data D1 by the selection processing unit 62 is repeated every time the reception processing unit 61 receives input data D. The selected data D1 is an example of the "first input data".
[0033] The process of randomly selecting input data D is realized not only as a process of directly selecting N pieces of selection data D1 from a plurality of input data D, but also as a first process of selecting a user Ua or a second process of selecting a terminal device 22. The first process is a process in which the selection processing unit 62 randomly selects N users Ua from a plurality of users Ua registered in advance, and selects N pieces of input data D corresponding to the N users Ua as selection data D1. The second process is a process in which the selection processing unit 62 randomly selects N terminal devices 22 from a plurality of terminal devices 22 registered in advance, and selects the N pieces of input data D received from the N terminal devices 22 as selection data D1.
[0034] The playback control unit 63 in FIG. 5 causes the playback system 13 in the facility 200 to play sounds (hereinafter referred to as "selected sounds") corresponding to the N pieces of selection data D1 selected by the selection processing unit 62. The selected sounds are played in parallel with the progress of the event. Therefore, users Ub (contestants, performers, organizers or spectators) in the facility 200 hear the selected sounds. As can be understood from the above explanation, the playback control unit 63 causes the selected sounds to be played for spectators of the event. The selected sounds are one example of a "first sound."
[0035] On the other hand, sounds corresponding to M pieces of input data D (hereinafter referred to as "non-selected data D2") not selected by the selection processing unit 62 among the multiple input data D received by the reception processing unit 61 are not reproduced (M is a natural number equal to or greater than 1). In other words, the M pieces of non-selected data D2 do not contribute to reproduction by the reproduction system 13. As described above, the M pieces of non-selected data D2 other than the N pieces of selected data D1 among the multiple input data D are ignored in reproducing the selected sounds. Therefore, the processing load on the reproduction control unit 63 can be reduced compared to a configuration in which sounds corresponding to the M pieces of non-selected data D2 are also reproduced.
[0036] The playback control unit 63 of the first embodiment generates an audio signal X by performing audio processing on N pieces of selection data D1, and supplies the audio signal X to the playback system 13, thereby causing the playback system 13 to play back the selected sound. That is, the selected sound generated by performing audio processing on N pieces of input sound represented by different selection data D1 is played back by the playback system 13.
[0037] The acoustic processing is a mixing process including a processing process and a mixing process. The processing process is a signal process for processing the input sounds represented by each of the N pieces of selection data D1. Specifically, the processing process includes a process for adjusting the timbre of each input sound, a process for controlling the localization of each input sound, a process for applying various sound effects to each input sound, or a process for adjusting the volume ratio between the N pieces of input sounds. The mixing process is a signal process for mixing the N pieces of input sounds after the processing process. By the acoustic processing exemplified above, an acoustic signal X of a selected sound that allows a listener to clearly distinguish each of the N pieces of input sounds is generated.
[0038] 6 is a flowchart illustrating a specific procedure of a process (hereinafter referred to as a "regeneration control process") Sa executed by the control device 51. The regeneration control process Sa is repeated, for example, at a predetermined cycle.
[0039] When the playback control process Sa is started, the control device 51 (reception processing unit 61) receives a plurality of input data D transmitted from different terminal devices 22 by the communication device 53 (Sa1). The control device 51 (selection processing unit 62) randomly selects N pieces of selection data D1 from the plurality of input data D (Sa2). The control device 51 (playback control unit 63) causes the playback system 13 to play selected sounds corresponding to the N pieces of selection data D1 (Sa3).
[0040] As can be understood from the above description, in the first embodiment, N pieces of selection data D1 are selected from a plurality of input data D according to input by users Ua of different terminal devices 22, and selected sounds according to the N pieces of selection data D1 are reproduced in parallel with the progress of the event. Therefore, compared to a configuration in which sounds according to all of the input data D received by the reception processing unit 61 are reproduced, it is easier for the listener to auditorily identify the selected sounds (and further each input sound). Also, in the first embodiment, the reproduced sounds according to the N pieces of selection data D1 are reproduced in the facility 200 where the event is held. Therefore, a user Ua of a terminal device 22 located remotely from the facility 200 can be involved in the atmosphere of the event in the facility 200.
[0041] In the first embodiment, sound processing including processing and mixing is performed on the N pieces of selection data D1. Therefore, compared with a configuration in which a mixed sound of input voices represented by the N pieces of selection data D1 is reproduced, there is an advantage that the listener can easily and clearly hear the input voice uttered by the user Ua of each terminal device 22.
[0042] B: Second embodiment A second embodiment will be described. Note that, for elements having the same functions as those in the first embodiment in each of the following exemplary aspects, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.
[0043] 7 is a block diagram illustrating a functional configuration of the control system 14 in the second embodiment. The storage device 52 in the second embodiment stores condition data Q. The condition data Q is data that specifies a condition for the selection processing unit 62 to select N pieces of selection data D1. Specifically, the condition data Q is a list in which a plurality of registered character strings Wr (Wr1, Wr2, ...) are registered.
[0044] The control device 51 of the second embodiment functions as a voice recognition unit 64 in addition to the same elements as those of the first embodiment (a reception processing unit 61, a selection processing unit 62, and a playback control unit 63). The voice recognition unit 64 identifies a spoken character string Wi by performing voice recognition on each of the multiple input data D received by the reception processing unit 61. The spoken character string Wi is a character string corresponding to the spoken content of the input voice represented by the input data D. Any known technology may be adopted for the voice recognition process for identifying the spoken character string Wi from the input data D.
[0045] The selection processing unit 62 of the second embodiment selects N pieces of selection data D1 from a plurality of input data D under conditions defined by the condition data Q. Specifically, the selection processing unit 62 selects, as the selection data D1, the input data D in which a spoken character string Wi correlating with any of a plurality of registered character strings Wr registered in the condition data Q is specified from the plurality of input data D. The correlation between the registered character string Wr and the spoken character string Wi is a relationship in which the registered character string Wr and the spoken character string Wi are similar to or match each other. That is, the selection processing unit 62 selects, as the selection data D1, the input data D in which a spoken character string Wi similar to or matching any of a plurality of registered character strings Wr is specified. Therefore, the number N of selection data D1 is a variable value according to the contents of speech by each user Ua. That is, the more users Ua who pronounce a phrase similar to or matching the registered character string Wr, the more the number N of selection data D1 increases.
[0046] The operation of having the reproduction system 13 reproduce the selected sound corresponding to the N pieces of selection data D1 selected by the selection processing unit 62 is the same as in the first embodiment. As can be understood from the above explanation, the utterance content permitted to be reproduced by the reproduction system 13 is registered in the condition data Q. In other words, the condition data Q specifies the conditions for the utterance content of the input voice permitted to be reproduced.
[0047] 8 is a flowchart illustrating a specific procedure of the regeneration control process Sb in the second embodiment. The regeneration control process Sb is repeated, for example, at a predetermined cycle.
[0048] When the playback control process Sb is started, the control device 51 (reception processing unit 61) receives a plurality of input data D by the communication device 53 in the same manner as in the first embodiment (Sb1). The control device 51 (voice recognition unit 64) identifies a spoken character string Wi of each of the plurality of input data D by a voice recognition process for each of the plurality of input data D (Sb2). The control device 51 (selection processing unit 62) selects the input data D in which the spoken character string Wi correlating with the registered character string Wr is identified as selected data D1 (Sb3). The control device 51 (playback control unit 63) causes the playback system 13 to play a selected sound corresponding to the N pieces of selected data D1 in the same manner as in the first embodiment (Sb4).
[0049] The second embodiment achieves the same effect as the first embodiment. In the second embodiment, the input data D representing the input voice of the speech content correlated with the registered character string Wr is selected as the selection data D1. Therefore, the possibility that a selection sound including an inappropriate character string is reproduced can be reduced.
[0050] In the above description, the input data D of the utterance content correlated with the registered character string Wr is selected, but the selection processing unit 62 may select the input data D of the utterance content different from the multiple registered character strings Wr registered in the condition data Q as the selection data D1. That is, the selection processing unit 62 selects the input data D in which the utterance character string Wi different from the registered character string Wr in the condition data Q is identified as the selection data D1. Therefore, the input data D corresponding to the utterance character string Wi correlated with the registered character string Wr is not selected. As can be understood from the above description, the condition data Q specifies either the condition of the input data D that is permitted to be selected by the selection processing unit 62 or the condition of the input data D that is prohibited from being selected by the selection processing unit 62.
[0051] C: Third embodiment Fig. 9 is an explanatory diagram regarding the operation of the playback control unit 63 in the third embodiment. As illustrated in Fig. 9, the playback system 13 in the third embodiment includes K playback units 16_1 to 16_K installed at different locations in a facility 200. Each playback unit 16_k (k=1 to K) is a speaker device that plays a selected sound represented by an audio signal X_k.
[0052] The reproduction control unit 63 of the third embodiment generates K-channel acoustic signals X_1 to X_K from the N pieces of selection data D1 selected by the selection processing unit 62. Each acoustic signal X_k corresponds to one of K sets (hereinafter referred to as "selection data sets") into which the N pieces of selection data D1 are divided. For example, assuming that the number N of selection data D1 is an integer multiple (z times) of the total number K of acoustic signals X_k (N=z·K), each of the K selection data sets is composed of z pieces of selection data D1. The reproduction control unit 63 generates an acoustic signal X_k from z pieces of selection data D1 constituting the k-th selection data set. Specifically, the reproduction control unit 63 generates an acoustic signal X_k representing a selection sound by executing acoustic processing (processing processing and mixing processing) on the z pieces of selection data D1 of the k-th selection data set. Note that the number z of selection data D1 used to generate each acoustic signal X_k may be different for each acoustic signal X_k.
[0053] Each acoustic signal X_k is supplied to one reproduction unit 16_k among the K reproduction units 16_1 to 16_K. Each reproduction unit 16_k reproduces a selected sound represented by the acoustic signal X_k. Specifically, the reproduction unit 16_k reproduces a selected sound corresponding to z pieces of selection data D1 constituting the k-th selection data set among the K selection data sets. Assuming that the number N of selection data D1 matches the total number K of acoustic signals X_k (z=1), each of the K reproduction units 16_1 to 16_K reproduces a selected sound corresponding to one different selection data D1.
[0054] Now, attention is paid to two arbitrary reproduction units 16_k1 and 16_k2 (k1, k2=1 to K, k1≠k2). An acoustic signal X_k1 is supplied to the reproduction unit 16_k1, and an acoustic signal X_k2 is supplied to the reproduction unit 16_k2. The reproduction unit 16_k1 reproduces a selected sound corresponding to z pieces of selection data D1 constituting the k1-th selection data set. Similarly, the reproduction unit 16_k2 reproduces a selected sound corresponding to z pieces of selection data D1 constituting the k2-th selection data set. That is, the reproduction control unit 63 causes the reproduction unit 16_k1 to reproduce a selected sound corresponding to a part (the k1-th selection data set) of the N pieces of selection data D1. The reproduction control unit 63 also causes the reproduction unit 16_k2 to reproduce a selected sound corresponding to the other part (the k2-th selection data set) of the N pieces of selection data D1. The playback unit 16_k1 is an example of a "first playback unit," and the playback unit 16_k2 is an example of a "second playback unit."
[0055] The third embodiment also achieves the same effect as the first embodiment. Moreover, in the third embodiment, a selection sound corresponding to some of the N pieces of selection data D1 and a selection sound corresponding to the other part of the selection data D1 are reproduced by separate reproduction units 16. Therefore, compared with a configuration in which selection sounds corresponding to the N pieces of selection data D1 are reproduced by a single reproduction unit 16, there is an advantage that the listener can easily and clearly hear the input voice pronounced by the user Ua of each terminal device 22. Note that the configuration of the second embodiment in which the condition data Q is used to select the N pieces of selection data D1 may be applied to the third embodiment.
[0056] D: Fourth embodiment In the first to third embodiments, a configuration has been exemplified in which the playback system 13 plays selected sounds corresponding to the N pieces of selection data D1. The playback control unit 63 of the fourth embodiment plays sounds corresponding to the M pieces of non-selected data D2 (hereinafter referred to as "non-selected sounds") in addition to the selected sounds corresponding to the N pieces of selection data D1. As described above, the M pieces of non-selected data D2 are input data D other than the N pieces of selection data D1 among the multiple pieces of input data D received by the reception processing unit 61. Note that the M pieces of non-selected data D2 may be a part of the multiple pieces of input data D other than the N pieces of selection data D1. The non-selected data D2 is an example of the "second input data".
[0057] 10 is a block diagram illustrating the configuration of the playback control unit 63 in the fourth embodiment. The playback control unit 63 in the fourth embodiment includes a first processing unit 631, a second processing unit 632, and a mixer 633.
[0058] The first processing unit 631 generates an audio signal Y1 by performing a first audio process on the N pieces of selection data D1. The first audio process is a mixing process including a processing process and a mixing process, similar to the audio process exemplified in the first embodiment. Therefore, the audio signal Y1 in the fourth embodiment corresponds to the audio signal X in the first embodiment. That is, the first audio process by the first processing unit 631 generates an audio signal Y1 of selected sounds that allows the listener to clearly distinguish each of the N pieces of input sounds.
[0059] The second processing unit 632 generates an audio signal Y2 by performing a second audio process on the M pieces of non-selected data D2. The second audio process is a signal process that reduces the clarity of the input sound represented by each piece of non-selected data D2. That is, the non-selected sound represented by the audio signal Y2 is a sound whose phoneme (i.e., the speech content) cannot be audibly identified by the listener. Specifically, the second audio process includes various processes such as a process of reducing the volume of the input sound, a process of mixing a noise component whose phoneme cannot be identified (e.g., white noise) with the input sound, a process of converting or replacing the input sound with a noise component whose phoneme cannot be identified, or a process of reducing multiple harmonic components of the input sound. That is, the second audio process by the second processing unit 632 generates an audio signal Y2 of non-selected sounds whose M pieces of input sound cannot be clearly identified by the listener.
[0060] The mixer 633 generates an audio signal X by mixing the audio signal Y1 generated by the first processing unit 631 and the audio signal Y2 generated by the second processing unit 632. For example, the mixer 633 generates the audio signal X by a weighted sum of the audio signals Y1 and Y2. The weighting value of the audio signal Y2 may be set to a smaller value than the weighting value of the audio signal Y1.
[0061] The audio signal X generated by the mixer 633 is supplied to the playback system 13. That is, a clear selected sound represented by the audio signal Y1 and an unclear non-selected sound represented by the audio signal Y2 are played back in the facility 200. As can be understood from the above description, the playback control unit 63 of the fourth embodiment causes the playback system 13 to play back selected sounds corresponding to the N pieces of selection data D1 and non-selected sounds corresponding to the M pieces of non-selected data D2.
[0062] The fourth embodiment also achieves the same effects as the first embodiment. In addition, in the fourth embodiment, non-selected sounds corresponding to M non-selected data D2 are reproduced together with selected sounds corresponding to N pieces of selection data D1. Therefore, it is possible to increase the number of users Ua who can participate in the atmosphere of the event in the facility 200. In addition, since the second sound processing that reduces the clarity of the M pieces of non-selected data D2 is executed, a state in which the selected sounds corresponding to the N pieces of selection data D1 can be easily heard by the listener is maintained. The configuration of the second or third embodiment may be applied to the fourth embodiment.
[0063] E: Fifth embodiment 11 is a block diagram illustrating a partial configuration of an information system 100 in the fifth embodiment. The configurations and operations of the reception processing unit 61 and the selection processing unit 62 in the fifth embodiment are similar to those in the first embodiment. For example, the selection processing unit 62 selects N pieces of selection data D1 from the multiple input data D received by the reception processing unit 61.
[0064] The playback control unit 63 of the control system 14 in the fifth embodiment generates an acoustic signal X from the N pieces of selection data D1 selected by the selection processing unit 62, and transmits the acoustic signal X to the distribution system 12. The acoustic signal X is transmitted from the control system 14 to the distribution system 12, for example, via a communication network 300. The acoustic processing for generating the acoustic signal X from the N pieces of selection data D1 is similar to that in the first embodiment. Note that the playback control unit 63 transmits the acoustic signal X to the distribution system 12 in parallel with supplying the acoustic signal X to the playback system 13. However, the supply of the acoustic signal X to the playback system 13 may be omitted. In other words, the playback system 13 may be omitted.
[0065] The delivery system 12 synthesizes the sound signal X received from the control system 14 into the sound data C2 of the content C. The mixed sound data C2 represents a mixed sound of the sound collected by the recording system 11 in the facility 200 and the selected sound corresponding to the N pieces of selection data D1. The sound emitting device 32 of the playback device 21 or the playback device 45 reproduces the sound represented by the sound data C2 received from the delivery system 12. That is, the selected sound corresponding to the N pieces of selection data D1 is reproduced from the sound emitting device 32 together with the sound collected in the facility 200. As can be understood from the above description, the reproduction control unit 63 of the fifth embodiment causes the device (playback device 21 or the playback device 45) used by the viewer of the content C to reproduce the selected sound. That is, the input voice pronounced by the user Ua is reproduced by the sound emitting device 32 (playback device 21 or the playback device 45) of the user Ua.
[0066] 12 is a flowchart illustrating a specific procedure of the regeneration control process Sc in the fifth embodiment. As in the first embodiment, the regeneration control process Sc is repeated, for example, at a predetermined cycle.
[0067] When the playback control process Sc is started, the control device 51 (reception processing unit 61) receives a plurality of input data D transmitted from different terminal devices 22 by the communication device 53, as in the first embodiment (Sc1). The control device 51 (selection processing unit 62) selects N pieces of selection data D1 from the plurality of input data D, as in the first embodiment (Sc2). The control device 51 (playback control unit 63) transmits an audio signal X corresponding to the N pieces of selection data D1 to the distribution system 12, thereby causing the playback device 21 or the playback device 45 to play the selected sounds corresponding to the N pieces of selection data D1 (Sc3).
[0068] As illustrated above, in the fifth embodiment, N pieces of selection data D1 are selected from a plurality of input data D according to inputs by users Ua of different terminal devices 22, and selected sounds according to the N pieces of selection data D1 are reproduced in parallel with the progress of the event. Therefore, as in the first embodiment, it is easier for the listener to auditorily identify the selected sounds compared to the configuration in which sounds according to all of the input data D received by the reception processing unit 61 are reproduced. Also, in the fifth embodiment, the reproduced sounds according to the N pieces of selection data D1 are reproduced by the device (the reproduction device 21 or the reproduction device 45) used by the viewer of the content C. Therefore, the user Ua of the terminal device 22 located remotely from the facility 200 can be involved in the atmosphere of the content C reproduced by the reproduction device 21 or the reproduction device 34.
[0069] In the above description, the fifth embodiment has been illustrated based on the configuration of the first embodiment, but the configurations of the second to fourth embodiments may be applied to the fifth embodiment. For example, the configuration of the second embodiment in which the selection processing unit 62 selects N pieces of selection data D1 by referring to the condition data Q is similarly applied to the fifth embodiment. Also, the configuration of the third embodiment in which the reproduction control unit 63 generates K-channel acoustic signals X_1 to X_K is also applied to the fifth embodiment. The selected sound represented by each acoustic signal X_k is reproduced by any one of K reproduction units (e.g., speaker devices) mounted on the reproduction device 21 or the reproduction device 45. Also, the configuration of the fourth embodiment may be applied to the fifth embodiment. For example, an acoustic signal X representing a mixed sound of the selected sound corresponding to the N pieces of selection data D1 and the non-selected sound corresponding to the M pieces of non-selected data D2 is transmitted to the distribution system 12, and the acoustic signal X is synthesized with the acoustic data C2. The sound emitting device 32 of the playback device 21 or the playback device 45 plays back selected sounds corresponding to the N pieces of selected data D1 and non-selected sounds corresponding to the M pieces of non-selected data D2.
[0070] F: Sixth embodiment 13 is a block diagram illustrating a functional configuration of a control system 14 in the sixth embodiment. A control device 51 in the control system 14 in the sixth embodiment executes a program stored in a storage device 52 to realize a selection processing unit 71, a reception processing unit 72, and a playback control unit 73.
[0071] The selection processing unit 71 selects N terminal devices (hereinafter referred to as "selected terminals") 22 from among the multiple terminal devices 22 registered as destinations of the service provided by the information system 100. The N selected terminals 22 are a part of the multiple terminal devices 22. The method by which the selection processing unit 71 selects the N selected terminals 22 is arbitrary. For example, the selection processing unit 71 randomly selects the N selected terminals 22 from the multiple terminal devices 22. The selection processing unit 71 also selects the N selected terminals 22 that satisfy a predetermined condition from among the multiple terminal devices 22. The predetermined condition is arbitrary, but is assumed to be a condition related to the communication environment, such as a communication speed exceeding a threshold. The selection processing unit 71 transmits control data G representing selection / non-selection to each of the multiple terminal devices 22. Note that the control data G representing selection may be transmitted only to the N selected terminals 22 from among the multiple terminal devices 22.
[0072] As in the first embodiment, each of the N selection terminals 22 transmits input data D corresponding to an input by the user Ua to the control system 14. Specifically, sound data representing a waveform of the voice input by the user Ua is transmitted from each selection terminal 22 to the control system 14 as the input data D. The transmission of the input data D by each selection terminal 22 is executed sequentially in parallel with the progress of the event in the facility 200.
[0073] On the other hand, input data D is not transmitted from M terminal devices 22 (hereinafter referred to as "non-selected terminals") 22 other than the N selected terminals 22 among the multiple terminal devices 22. That is, each non-selected terminal 22 does not transmit input data D even when it receives voice input from the user Ua. Therefore, N pieces of input data D corresponding to different selected terminals 22 are transmitted sequentially to the control system 14 in parallel with the progress of the event. That is, in the sixth embodiment, input data D is transmitted from one or more terminal devices 22 (selected terminals 22) that are a part of the multiple terminal devices 22.
[0074] The reception processing unit 72 receives the input data D transmitted from each of the N selected terminals 22 via the communication device 53. The reception of the N pieces of input data D by the reception processing unit 61 is repeated at a predetermined cycle in parallel with the progress of the event.
[0075] The playback control unit 73 causes the playback system 13 to play a selected sound corresponding to the N pieces of input data D received by the reception processing unit 72. Specifically, the playback control unit 73 generates an audio signal X by acoustic processing on the N pieces of input data D, and supplies the audio signal X to the playback system 13 to play the selected sound. That is, the selected sound generated by acoustic processing on the N pieces of input sounds represented by different input data D is played by the playback system 13. The acoustic processing is a mixing process including processing and mixing, as in the first embodiment. A user Ub in the facility 200 hears the selected sound in parallel with the progress of the event.
[0076] 14 is a flowchart illustrating a specific procedure of the playback control process Sd executed by the control device 51. After the selection of the selected terminal 22 by the selection processing unit 71 and the transmission of the control data G, the playback control process Sd is repeated, for example, at a predetermined cycle.
[0077] When the playback control process Sd is started, the control device 51 (reception processing unit 72) receives N pieces of input data D transmitted from different selected terminals 22 via the communication device 53 (Sd1). The control device 51 (playback control unit 73) causes the playback system 13 to play selected sounds corresponding to the N pieces of input data D (Sd2).
[0078] As can be understood from the above explanation, in the sixth embodiment, input data D is received from each of the N selected terminals 22 that are part of the multiple terminal devices 22, and a selected sound corresponding to the N pieces of input data D is reproduced in parallel with the progress of the event. Therefore, it is easier for the listener to auditorily identify the selected sound, compared to a configuration in which a selected sound corresponding to the input data D transmitted from all of the multiple terminal devices 22 is reproduced. In addition, since the non-selected terminals 22 do not transmit input data D, there is also the advantage that the amount of communication required to reproduce the selected sound can be reduced.
[0079] Note that the same configuration as in the third embodiment is also applied to the sixth embodiment. That is, the playback control unit 73 generates K-channel acoustic signals X_1 to X_K from the N pieces of input data D received by the reception processing unit 72. The same processing as in the third embodiment is used to generate the acoustic signals X_1 to X_K. Each acoustic signal X_k is supplied to one playback unit 16_k among the K playback units 16_1 to 16_K included in the playback system 13. That is, each playback unit 16_k plays a selected sound represented by the acoustic signal X_k.
[0080] The same configuration as in the fifth embodiment is also applied to the sixth embodiment. That is, the playback control unit 73 transmits the audio signal X generated from the N pieces of input data D to the distribution system 12. The distribution system 12 synthesizes the audio signal X received from the control system 14 with the audio data C2 of the content C. Therefore, the sound emitting device 32 of the playback device 21 or the playback device 45 reproduces the selected sound according to the N pieces of input data D. That is, the playback control unit 73 of the sixth embodiment causes the device (the playback device 21 or the playback device 45) used by the viewer of the content C to reproduce the selected sound. As can be understood from the above explanation, the input voice pronounced by the user Ua is reproduced by the sound emitting device 32 (the playback device 21 or the playback device 45) of the user Ua.
[0081] G: Variation Specific modified embodiments that can be added to each of the above-mentioned embodiments are illustrated below. A plurality of embodiments arbitrarily selected from the following examples may be appropriately combined as long as they are not mutually contradictory.
[0082] (1) The method by which the selection processing unit 62 selects N pieces of selection data D1 from a plurality of input data D is not limited to the methods exemplified in the above-mentioned embodiments. Examples of the method by which the selection processing unit 62 selects N pieces of selection data D1 include, for example, the following aspects 1A to 1D.
[0083] [Aspect 1A] In the aspect 1A, the selection processing unit 62 selects the input data D of a specific user Ua from among the multiple users Ua. Specifically, the identification information of the user Ua (or the identification information of the terminal device 22 of the user Ua) that satisfies a specific condition from among the multiple users Ua is stored in the storage device 52. For example, the identification information of the user Ua who has completed payment of a predetermined amount by electronic payment processing, or the identification information of the user Ua who has consumed a predetermined amount of points used for services by the information system 100 is stored in the storage device 52. The selection processing unit 62 selects, as the selected data D1, the input data D received from the terminal device 22 of the user Ua whose identification information is stored in the storage device 52 from among the multiple input data D received by the reception processing unit 61. In addition, the number of times that the input data D of the user Ua is selected as the selected data D1 within a unit time may be changed for each user Ua according to the payment amount or the amount of points consumed by the electronic payment processing.
[0084] [Aspect 1B] In aspect 1B, the selection processing unit 62 selects the input data D of each user Ua according to the attributes of each user Ua. For example, a plurality of types of attributes are stored in the storage device 52 for each of the plurality of users Ua. The attributes of the user Ua may be, for example, age or gender. In a sporting event, the team that the user Ua supports may be registered as the attribute of the user Ua. The selection processing unit 62 selects, as the selection data D1, the input data D of a user Ua with a specific attribute registered among the plurality of users Ua. The selection processing unit 62 may select the input data D of a user Ua who supports either the home team or the away team in a sporting event. The selection processing unit 62 may change the attributes of the user Ua that are the selection target of the input data D over time.
[0085] Furthermore, the selection processing unit 62 may select N pieces of selection data D1 so that the number of pieces of selection data D1 selected from the multiple input data D of the user Ua with the first attribute and the number of pieces of selection data D1 selected from the multiple input data D of the user Ua with the second attribute have a predetermined ratio. For example, the selection processing unit 62 selects N pieces of selection data D1 so that the number of pieces of selection data D1 corresponding to the user Ua who supports the home team and the number of pieces of selection data D1 corresponding to the user Ua who supports the away team have a predetermined ratio.
[0086] [Aspect 1C] In aspect 1C, the selection processing unit 62 selects N pieces of selection data D1 depending on the time point when the input data D is received from the terminal device 22 of each user Ua. For example, in each of a plurality of periods on the time axis, the selection processing unit 62 selects N pieces of selection data D1 from among the plurality of input data D received by the reception processing unit 61 in the order of reception within that period.
[0087] [Aspect 1D] In aspect 1D, the selection processing unit 62 selects the input data D manually designated by the administrator from among the multiple input data D as the selected data D1. The administrator is, for example, an event organizer or an administrator of the information system 100. For example, the administrator visually recognizes the speech string Wi identified from each input data D, and designates the input data D from among the multiple input data D whose speech string Wi satisfies a specific condition. For example, the input data D corresponding to an appropriate speech string Wi is designated. The selection processing unit 62 selects the input data D designated by the administrator from among the multiple input data D as the selected data D1.
[0088] Note that the N pieces of selection data D1 may be selected by a combination of two or more methods selected from the above aspects (first embodiment to fifth embodiment and aspect 1A to aspect 1D). For example, the selection processing unit 62 may randomly select N pieces of selection data D1 from the multiple pieces of selection data D1 selected by one or more methods among aspects 1A to 1D. Also, the same method as the selection of the selection data D1 exemplified in each of the above aspects is applied to the selection of the N selection terminals 22 in the sixth embodiment.
[0089] (2) In each of the above-mentioned embodiments, the acoustic data representing the waveform of the input sound to the sound collection device 46 is exemplified as the input data D, but the format of the input data D (selected data D1 or non-selected data D2) is not limited to the above-mentioned examples. Examples of the format of the input data D other than the formats exemplified in each of the above-mentioned embodiments include, for example, the following aspects 2A to 2C.
[0090] [Aspect 2A] The input data D in aspect 2A is character string data representing a character string designated by the user Ua. For example, the input data D representing a character string designated by the user Ua through an operation on the operation device 44 is transmitted from the terminal device 22 to the control system 14. The playback control unit 63 of the control system 14 generates a synthetic signal by performing a voice synthesis process on the selection data D1, and generates an audio signal X from the N-channel synthetic signals generated from the different selection data D1. Any known technology may be used for the voice synthesis process. According to aspect 2A, a selection sound representing the character string designated by each user Ua is played back.
[0091] [Aspect 2B] The input data D in aspect 2B is instruction data instructing one of a plurality of sounds (hereinafter referred to as "registered sounds") registered in advance. The registered sounds are various sounds that may be produced at an event, such as cheers or curses. For example, the input data D indicating the registered sound selected by the user Ua through an operation on the operation device 44 is transmitted from the terminal device 22 to the control system 14. The playback control unit 63 of the control system 14 acquires from the storage device 52 a registered signal of the registered sound indicated by the selection data D1 among a plurality of registered signals representing waveforms of different registered sounds, and generates an audio signal X from the N-channel registered signals acquired for the different selection data D1. According to aspect 2B, the registered sound designated by each user Ua is reproduced.
[0092] [Aspect 2C] In each of the above-mentioned embodiments, the input data D representing the input voice uttered by the user Ua is transmitted from the terminal device 22 to the control system 14 in parallel with the progress of the event. In embodiment 2C, for example, sound data generated by sound collection by the sound collection device 46 before the start of the event is used as the input data D in parallel with the progress of the event. For example, sound data representing different input voices is stored in the storage device 42 of the terminal device 22. The control device 41 transmits the sound data selected by the user Ua by operating the operation device 44 as the input data D from the communication device 43 to the control system 14. The control device 41 may edit or process the sound data stored in the storage device 42 in response to an instruction from the user Ua. The control device 41 may automatically edit or process the sound data using a known sound processing technique.
[0093] The input data D exemplified in each of the above-mentioned embodiments and the input data D exemplified in aspects 2A to 2C are collectively expressed as data indicating a voice corresponding to an input by the user Ua. The input by the user Ua is, for example, a voice input using the sound pickup device 46, or a manual input using the operation device 44. However, the sound represented by the input data D is not limited to a voice (i.e., a human voice). For example, 、The sound represented by the input data D may be a sound other than speech, such as a clap, a whistle, or a musical instrument sound. Therefore, the input data D is comprehensively expressed as data representing a sound corresponding to an input by the user Ua.
[0094] (3) In the fifth embodiment, the sound signal X corresponding to the N pieces of selection data D1 is synthesized into the sound data C2 of the content C, but the configuration and operation of synthesizing the sound signal X into the sound data C2 are not essential. For example, the playback control unit 63 of the control system 14 may directly transmit the sound signal X to the terminal device 22 via the communication network 300. The control device 41 of the terminal device 22 causes the sound emitting device 32 of the playback device 45 to play the selection sound represented by the sound signal X received by the communication device 43. The playback control unit 63 in one aspect including the fifth embodiment and the third modification example is only required to operate such that the device used by the viewer of the content C (i.e., the video of the event) plays the selection sound, and the specific configuration and operation for the playback of the selection sound by the device are arbitrary. The playback device 21 and the playback device 45 are examples of devices used by the viewer.
[0095] (4) In each of the above-described embodiments, the selected sound is reproduced by the reproduction system 13, but a character string representing the selected sound (hereinafter referred to as a "selected character string") may be displayed, for example, on a display device (not shown) in the facility 200. For example, the reproduction control unit 63 estimates the selected character string by a voice recognition process for the acoustic signal X, and causes the display device to display the selected character string. The selected character string expressed in the first language may be translated into the second language by a known machine translation process and then displayed on the display device. In addition, the selected character string may be displayed on the display device 31 of the reproduction device 45 (or the display device 31 of the reproduction device 21) in the terminal device 22 by transmitting character string data representing the selected character string (first language or second language) from the control system 14 to the terminal device 22. Note that, in the above description, a form in which a selected character string representing the selected sound is displayed has been exemplified, but the reproduction control unit 63 may display an image (for example, an icon) that changes depending on the selected sound on the display device in the facility 200 or the display device 31.
[0096] (5) The number of times the input data D is received by the reception processing unit 61 (hereinafter referred to as the "number of receptions") may be displayed on a display device in the facility 200 or on the display device 31 of the user Ua. The number of receptions can also be expressed as the total number of times that a plurality of users Ua speak the input voice. The reception processing unit 61 counts the number of receptions in parallel with the reception of the input data D, and the playback control unit 63 displays the number of receptions on a display device in the facility 200 or on the display device 31 of the user Ua.
[0097] (6) In the second embodiment, the condition data Q used to select the N pieces of selection data D1 may be switched sequentially. Specifically, the selection processing unit 62 selects each of the multiple pieces of condition data Q stored in the storage device 52 in sequence in parallel with the progress of the event, and selects the N pieces of selection data D1 using the selected condition data Q. With the above configuration, the spoken string Wi that is permitted to be reproduced as a selected sound changes over time.
[0098] Specifically, the selection processing unit 62 changes the condition data Q for each scene of the event. For example, assume a sporting event in which a first team and a second team compete. During a period in which the first team is on offense, the selection processing unit 62 activates the condition data Q in which the player names of the first team are registered as the registered character string Wr. Therefore, the player names of the first team pronounced by the user Ua are likely to be played as the selected sound. On the other hand, during a period in which the second team is on offense, the selection processing unit 62 activates the condition data Q in which the player names of the second team are registered as the registered character string Wr. Therefore, the player names of the second team pronounced by the user Ua are likely to be played as the selected sound.
[0099] As can be understood from the above examples, by changing the condition data Q over time, an appropriate selected sound can be played for each scene of the event. The selection processing unit 62 identifies the scene of the event using a known scene identification technique that analyzes the video recorded by the recording system 11. Also, the manager of the control system 14 may manually specify the scene in parallel with the progress of the event.
[0100] (7) In each of the above-described embodiments, the selection processing unit 62 selects multiple pieces of selection data D1 (N≧2), but the selection processing unit 62 may select one piece of selection data D1 (N=1). In the above-described first embodiment, the number N of pieces of selection data D1 is a fixed value, but as can be understood from the example of the second embodiment, the number N may be a variable value. For example, the total number of pieces of input data D received by the reception processing unit 61 may be multiplied by a predetermined coefficient (e.g., a number less than 1) and set as the number N of pieces of selection data D1.
[0101] (8) Although sporting events and music events are given as examples of the above-mentioned types of events, the events to which the above-mentioned types of events apply are not limited to these examples. For example, the above-mentioned types of events apply to various events held for a specific purpose, such as dance events performed by dancers, theater events featuring actors, lecture events where speakers give talks, and educational events where various educational institutions such as schools and cram schools provide lessons to students.
[0102] (9) As described above, the functions of the control system 14 exemplified above are realized by the cooperation of one or more processors constituting the control device 51 and the program stored in the storage device 52. The program can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that the non-transitory recording medium includes any recording medium except for a transient, propagating signal, and does not exclude volatile recording media. In addition, in a configuration in which a distribution device distributes a program via a communication network, the recording medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.
[0103] H: Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.
[0104] A playback control method according to one aspect (aspect 1) of the present disclosure receives a plurality of input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, selects one or more first input data that are part of the plurality of input data, and plays a first sound corresponding to the one or more first input data in parallel with the progress of the event. In the above aspect, one or more first input data are selected from the plurality of input data received from different terminal devices, and a first sound corresponding to the one or more first input data is played in parallel with the progress of the event. Therefore, it is easier for a listener to auditorily identify the first sound, compared to a configuration in which a first sound corresponding to all input data received from a plurality of terminal devices is played.
[0105] "Input data" is any data corresponding to an input by a user. Specifically, input data is data indicating a sound corresponding to an input by a user. For example, specific examples of "input data" include audio data indicating a waveform of an audio sound collected from a user by a sound collection device, character string data indicating a character string specified by a user, and selection data indicating a sound selected by a user from among a plurality of sounds. In a mode in which character string data is received as input data from a terminal device, for example, a first sound generated by a voice synthesis process for the character string data is reproduced. Also, in a mode in which selection data is received as input data from a terminal device, for example, a sound indicated by the selection data from among a plurality of sounds registered in advance is reproduced as the first sound.
[0106] The "process of selecting one or more first input data that are part of a plurality of input data" includes not only a mode in which a portion of the plurality of input data is directly selected as the first input data, but also a mode in which one or more terminal devices are selected from a plurality of terminal devices, and input data received from the one or more selected terminal devices among the plurality of input data is selected as the first input data (i.e., a mode in which input data is indirectly selected by selecting a terminal device).
[0107] The target for which the first sound is reproduced is, for example, a spectator or a related person of the event. A related person of the event is, for example, an administrator who manages and operates the event, or a participant who participates in the event (for example, a performer of a music event or a contestant of a sport event). Furthermore, an "event spectator" is a spectator who is actually present at the place where the event is held (hereinafter referred to as the "event location"), or a spectator who watches a video of the event (i.e., a viewer of the video). Reproduction of the first sound for a spectator or a related person who is present at the event location includes, for example, an operation of causing a playback system installed at the event location to reproduce the first sound, or an operation of causing a terminal device used by the spectator or a related person to reproduce the first sound. Furthermore, reproduction of the first sound for a spectator who watches a video of the event includes an operation of causing a device used by the spectator to reproduce the first sound. It does not matter whether the device used to watch the video is the same as the device that reproduces the first sound.
[0108] The process of "playing a first sound corresponding to one or more first input data" means that the first sound corresponding to one or more first input data is preferentially played back compared to a second sound corresponding to second input data other than the first input data. Therefore, "playing a first sound" includes an operation of selectively playing back only the first sound corresponding to one or more first input data out of a plurality of input data, as well as an operation of playing back the first sound corresponding to one or more first input data preferentially (for example, at a louder volume than the second sound) compared to a second sound corresponding to one or more second input data other than the first input data.
[0109] In a specific example (aspect 2) of aspect 1, the first sound is reproduced by a device used by a viewer of the video of the event. In the above aspect, the first sound corresponding to one or more first input data selected from a plurality of input data is reproduced by a device used by the viewer of the video. Therefore, for example, a user of a terminal device located remotely from the event location can be involved in the atmosphere of the video of the event.
[0110] A specific example of the process of "playing the first sound on a device used by a viewer of the event video" is, for example, an operation of transmitting sound data representing the first sound to the viewer's device to play the first sound on the viewer's device. Any method can be used to transmit the sound data to the viewer's device. For example, the sound data may be mixed with video data representing the video viewed by the viewer and transmitted to the device, or the sound data may be transmitted independently of the video data.
[0111] In a specific example (Aspect 3) of Aspect 1, the first sound is reproduced by a reproduction system installed at the location where the event is held. In the above aspect, the first sound corresponding to one or more first input data selected from a plurality of input data is reproduced at the location where the event is held, so that, for example, a user of a terminal device located remotely from the event location can be involved in the atmosphere of the event at the event location.
[0112] In a specific example (Aspect 4) of Aspect 3, the one or more first input data are two or more first input data, the reproduction system includes a first reproduction unit and a second reproduction unit, and in reproducing the first sound, the first reproduction unit reproduces the first sound corresponding to a part of the first input data among the two or more first input data, and the second reproduction unit reproduces the first sound corresponding to the other part of the first input data among the two or more first input data. In the above aspect, the first sound corresponding to the part of the first input data among the two or more first input data selected from a plurality of input data and the first sound corresponding to the other part of the first input data are reproduced by separate reproduction units. This has the advantage that the listener can easily and clearly hear the sound corresponding to the input by the user of each terminal device.
[0113] In a specific example (Aspect 5) of any one of Aspects 1 to 3, the one or more first input data are two or more first input data, a first sound processing including processing and mixing is executed on the two or more first input data, and in playing the first sound, the first sound after the first sound processing is played. In the above aspects, the first sound processing including processing and mixing is executed on the sound corresponding to the two or more first input data. This has the advantage that the listener can easily and clearly hear the sound corresponding to the input by the user of each terminal device.
[0114] In a specific example (Aspect 6) of any one of Aspects 1 to 5, a sound corresponding to one or more second input data other than the one or more first input data among the plurality of input data is not reproduced. In the above aspects, the second input data other than the first input data among the plurality of input data is ignored in reproducing the first sound. Therefore, the load of the process of reproducing the sound can be reduced.
[0115] In a specific example (aspect 7) of any one of aspects 1 to 5, a second sound processing is performed on one or more second input data other than the one or more first input data among the plurality of input data, to reduce the clarity of the data, and the first sound and the second sound after the second sound processing are played back in the playback of the first sound. In the above aspect, a second sound corresponding to one or more second input data other than the first input data is played back together with a first sound corresponding to one or more first input data among the plurality of input data. Therefore, it is possible to increase the number of users who can participate in the atmosphere of the event. In addition, a second sound processing is performed on one or more second input data to reduce the clarity of the data. Therefore, despite the configuration in which the first sound and the second sound are played back in parallel, a state in which the listener can easily hear the first sound is maintained.
[0116] The second acoustic processing is processing that reduces the discriminability of the speech content of the second sound corresponding to one or more second input data (the possibility that the listener can audibly discern the speech content). Specifically, the second acoustic processing includes various types of processing, such as processing that reduces the volume of the sound represented by the second input data, processing that mixes in a noise component whose phoneme cannot be discriminated (e.g., white noise), or processing that reduces multiple harmonic components of the input voice.
[0117] In a specific example (Aspect 8) of any one of Aspects 1 to 7, in the selection of the one or more first input data, the one or more first input data are randomly selected from the plurality of input data. According to the above aspect, the processing load of selecting the one or more first data from the plurality of input data is reduced.
[0118] In a specific example (Aspect 9) of any one of Aspects 1 to 7, the one or more first input data are selected by selecting, from among the plurality of input data, the one or more first input data representing sounds of speech content that are correlated with or different from a registered character string. According to the above aspect, it is possible to reduce the possibility that a first sound including an inappropriate character string is reproduced.
[0119] A control system according to one embodiment (embodiment 10) of the present disclosure includes a receiving processing unit that receives multiple input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, a selection processing unit that selects one or more first input data that are part of the multiple input data, and a playback control unit that plays a first sound corresponding to the one or more first input data in parallel with the progress of the event.
[0120] A program according to one embodiment (embodiment 11) of the present disclosure causes a computer to function as a receiving processing unit that receives multiple input data corresponding to inputs by users of different terminal devices in parallel with the progress of an event, a selection processing unit that selects one or more first input data that are part of the multiple input data, and a playback control unit that plays a first sound corresponding to the one or more first input data in parallel with the progress of the event. [Explanation of symbols]
[0121] 100...information system, 200...facility, 300...communication network, 11...recording system, 12...distribution system, 13...playback system, 14...control system, 21...playback device, 22...terminal device, 31...display device, 32...sound emission device, 41, 51...control device, 42, 52...storage device, 43, 53...communication device, 44...operation device, 45...playback device, 46...sound collection device, 61, 72...receiving processing unit, 62, 71...selection processing unit, 63, 73...playback control unit, 631...first processing unit, 632...second processing unit, 633...mixing unit, 64...voice recognition unit.
Claims
1. receiving a plurality of pieces of input data corresponding to inputs by users of different terminal devices in parallel with the progress of the event; Selecting one or more first input data that are part of the plurality of input data; performing a second acoustic process for reducing clarity of one or more second input data other than the one or more first input data among the plurality of input data; a first sound corresponding to the one or more first input data and a second sound having undergone the second sound processing are reproduced in parallel with the progress of the event; A computer-implemented playback control method.
2. In the reproduction of the first sound and the second sound, the first sound and the second sound are reproduced on a device used by a viewer of the video of the event. The playback control method according to claim 1.
3. In the reproduction of the first sound and the second sound, the first sound and the second sound are reproduced by a reproduction system installed at a place where the event is held. The playback control method according to claim 1.
4. The one or more first input data are two or more first input data, the playback system includes a first playback unit and a second playback unit; In reproducing the first sound and the second sound, causing the first reproduction unit to reproduce the first sound and the second sound corresponding to a part of the first input data among the two or more first input data; causing the second reproduction unit to reproduce the first sound and the second sound corresponding to another part of the first input data among the two or more first input data. The playback control method according to claim 3.
5. The one or more first input data are two or more first input data, performing a first sound processing including a processing process and a mixing process on the two or more first input data; In the reproduction of the first sound and the second sound, the first sound and the second sound that have been subjected to the first sound processing are reproduced. The playback control method according to any one of claims 1 to 3.
6. In the selection of the one or more first input data, the one or more first input data are randomly selected from the plurality of input data. The playback control method according to any one of claims 1 to 5.
7. In the selection of the one or more first input data, the one or more first input data representing a sound of a speech content that is correlated with or different from a registered character string is selected from the plurality of input data. The playback control method according to any one of claims 1 to 5.
8. a reception processing unit that receives a plurality of pieces of input data corresponding to inputs by users of different terminal devices in parallel with the progress of the event; a selection processing unit that selects one or more first input data that are part of the plurality of input data; a playback control unit that executes a second sound processing for reducing clarity of one or more second input data other than the one or more first input data among the plurality of input data, and plays a first sound corresponding to the one or more first input data and a second sound after the second sound processing in parallel with the progress of the event; A control system comprising:
9. a receiving and processing unit that receives a plurality of pieces of input data corresponding to inputs by users of different terminal devices in parallel with the progress of the event; a selection processing unit that selects one or more first input data that are part of the plurality of input data; and a playback control unit that executes a second sound processing for reducing clarity of one or more second input data other than the one or more first input data among the plurality of input data, and plays a first sound corresponding to the one or more first input data and a second sound after the second sound processing in parallel with the progress of the event. A program that makes a computer function as a
Citation Information
Patent Citations
Viewer participant type broadcast program distribution method, and transmission system and reception system adopting this method
JP2003116119A
Method for preventing prohibited word transmission, telephone for preventing prohibited word transmission, and server for preventing prohibited word transmission
JP2010103751A
Call speech processor and call speech controller and method
JP2011199550A
Information processing apparatus and method, program, and information processing system
JP2012129800A
Remote Engagement System
JP2015507855A
