Playback control method, control system, and program

Abstract

To enable a user to selectively or preferentially listen to a plurality of sounds collected in parallel with the progress of an event. [Solution] A signal acquisition unit 62 acquires multiple sound signals X1 to XN generated for an event by collecting sound at different positions in parallel with the progress of the event. An information acquisition unit 63 acquires, for each of multiple terminal devices, playback control information Q that specifies playback conditions for each of the multiple sound signals X1 to XN. A playback control unit 64 executes playback control processing for each of the multiple terminal devices. In the playback control processing, under the playback conditions specified by the playback control information Q for the terminal device, the playback device is caused to play back sound corresponding to one or more sound signals Xn of the multiple sound signals X1 to XN in parallel with the progress of the event.

Description

[Technical field]

[0001] The present disclosure relates to techniques for controlling sound reproduction. [Background technology]

[0002] For example, a technology has been proposed for distributing contents of various events, such as sporting events or music events, to a plurality of terminal devices in parallel with the progress of the events. For example, Patent Document 1 discloses a technology for distributing digital data including video and audio to terminal devices. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-87760 A Summary of the Invention [Problem to be solved by the invention]

[0004] The sound of the content delivered to the terminal device is the overall sound recorded at the location where the event is held. However, the sound that the user who watches the event actually wants to listen to varies depending on the preferences of each user. In consideration of the above circumstances, one aspect of the present disclosure aims to enable the user to selectively or preferentially listen to multiple sounds that are collected in parallel with the progress of the event. [Means for solving the problem]

[0005] In order to solve the above problems, a playback control method according to one embodiment of the present disclosure acquires a plurality of acoustic signals generated for an event by sound pickup at different positions in parallel with the progress of the event, acquires playback control information for each of a plurality of terminal devices specifying playback conditions for each of the plurality of acoustic signals, executes a playback control process for each of the plurality of terminal devices, and in the playback control process, causes a playback device to play sound corresponding to one or more of the plurality of acoustic signals in parallel with the progress of the event under playback conditions specified by the playback control information for the terminal device.

[0006] A control system according to one embodiment of the present disclosure includes a signal acquisition unit that acquires multiple acoustic signals generated for an event by sound pickup at different positions in parallel with the progress of the event, an information acquisition unit that acquires, for each of a plurality of terminal devices, playback control information that specifies playback conditions for each of the multiple acoustic signals, and a playback control unit that executes playback control processing for each of the multiple terminal devices, in which the playback control processing causes a playback device to play sound corresponding to one or more of the multiple acoustic signals in parallel with the progress of the event under playback conditions specified by the playback control information for the terminal device.

[0007] A program according to one embodiment of the present disclosure causes a computer to function as a signal acquisition unit that acquires multiple acoustic signals generated for an event by sound pickup at different positions in parallel with the progress of the event, an information acquisition unit that acquires, for each of a plurality of terminal devices, playback control information that specifies playback conditions for each of the multiple acoustic signals, and a playback control unit that executes playback control processing for each of the multiple terminal devices, wherein the playback control processing causes a playback device to play sounds corresponding to one or more of the multiple acoustic signals in parallel with the progress of the event under playback conditions specified by the playback control information of the terminal device. [Brief description of the drawings]

[0008] [Figure 1]1 is a block diagram illustrating a configuration of an information system in a first embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating a configuration of a playback device. [Diagram 3] FIG. 2 is a block diagram illustrating a configuration of a terminal device. [Figure 4] FIG. 2 is a block diagram illustrating a configuration of a control system. [Diagram 5] FIG. 2 is a block diagram illustrating a functional configuration of the control system. [Figure 6] FIG. 2 is a schematic diagram of playback control information. [Figure 7] FIG. 4 is a schematic diagram of an operation screen. [Figure 8] 13 is a flowchart of a control process. [Figure 9] 13 is a flowchart of a playback control process. [Figure 10] FIG. 11 is a block diagram illustrating a functional configuration of a control system according to a second embodiment. [Figure 11] 10 is a flowchart of a control process in a second embodiment. [Figure 12] FIG. 11 is an explanatory diagram regarding selection of playback control information in the second embodiment. [Figure 13] FIG. 13 is a block diagram illustrating a functional configuration of a control system according to a third embodiment. [Figure 14] FIG. 13 is a schematic diagram of playback control information in the fourth embodiment. [Figure 15] FIG. 13 is a block diagram illustrating the configuration of a sound emitting device in a fifth embodiment. [Figure 16] FIG. 13 is a schematic diagram of playback control information in the fifth embodiment. [Figure 17] FIG. 23 is an explanatory diagram of the operation of a playback control unit in the sixth embodiment. [Figure 18] 23 is a flowchart of a reproduction control process in the sixth embodiment. [Figure 19] FIG. 23 is a block diagram of an information system and a terminal device according to the seventh embodiment. [Figure 20]FIG. 23 is a block diagram illustrating a functional configuration of a terminal device in the seventh embodiment. [Figure 21] 13 is a flowchart of a control process in the seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A: First embodiment 1 is a block diagram illustrating the configuration of an information system 100 according to the first embodiment. The information system 100 is a computer system that provides a plurality of users Ua with services related to an event held in a facility 200. The event in the first embodiment is a sporting event in which a plurality of participants compete in various sports. Therefore, the facility 200 is, for example, a stadium or a gymnasium. In other words, the facility 200 corresponds to a place where the event is held.

[0010] Within the facility 200, there are a plurality of spectators Ub and a plurality of related parties Uc. Each spectator Ub watches the event within the facility 200. Note that due to various circumstances, such as preventing the spread of infectious diseases, the event may be held in a situation where no spectators Ub are present within the facility 200. The plurality of related parties Uc may be, for example, participants (players or coaches) who will take part in the event, referees who will proceed with the event, an administrator who will run the event, an administrator who will manage the facility 200, or commentators who will provide commentary or explanation of the event.

[0011] A plurality of users Ua are located outside the facility 200. Specifically, each user Ua is located at a location (e.g., at home) remote from the facility 200. However, the user Ua may be located within the facility 200. In other words, the user Ua may be a visitor Ub or a related person Uc.

[0012] Each user Ua can use a playback device 21 and a terminal device 22. The terminal device 22 is, for example, a portable information terminal such as a smartphone or a tablet terminal. Note that a stationary personal computer may also be used as the terminal device 22. The terminal device 22 can communicate with the information system 100 via a communication network 300 such as the Internet.

[0013] The reproducing device 21 reproduces the content C representing the status of an event held in the facility 200. FIG. 2 is a block diagram illustrating the configuration of the reproducing device 21. The content C is composed of video data C1 representing video and audio data C2 representing sound. The reproducing device 21 includes a display device 23 that displays the video represented by the video data C1, and an audio output device 24 that reproduces the sound represented by the audio data C2. For example, a television receiver that reproduces a television program transmitted by airwaves as the content C is used as the reproducing device 21. Also, a portable information terminal such as a smartphone or a tablet terminal is used as the reproducing device 21. A large-sized video device (public viewing) that can be viewed by many users Ua may be used as the reproducing device 21. The content C may be a radio program composed only of the audio data C2.

[0014] 1, the information system 100 includes a recording system 11, a distribution system 12, a sound collection system 13, and a control system 14. The recording system 11 and the sound collection system 13 are installed in a facility 200. The recording system 11 communicates with the distribution system 12. The sound collection system 13 communicates with the control system 14. The distribution system 12 and the control system 14 may be configured as a single device.

[0015] The recording system 11 records a video of an event held in the facility 200. Specifically, the recording system 11 includes an imaging device that captures video of the event and a sound collection device that collects sound of the event (not shown). The recording system 11 generates a video composed of the video captured by the imaging device and the sound collected by the sound collection device.

[0016] The distribution system 12 distributes content C including the video recorded by the recording system 11 to each of the multiple playback devices 21. The distribution system 12 can also distribute content C to each terminal device 22. Specifically, the distribution system 12 distributes content C in real time in parallel with the progress of the event in the facility 200 (i.e., live distribution). Each playback device 21 receives content C from the distribution system 12 via the communication network 300 and plays the content C in parallel with the progress of the event. The user Ua can grasp the status of the event in real time by watching the content C played by the playback device 21. As can be understood from the above explanation, the user Ua is a viewer of the content C related to the event held in the facility 200.

[0017] The sound collection system 13 is an acoustic system that collects sounds within the facility 200. The sound collection system 13 includes a plurality (N pieces) of sound collection units 15_1 to 15_N (N is a natural number of 2 or more) installed at different positions within the facility 200. Each sound collection unit 15_n (n=1 to N) is a microphone that collects surrounding sounds and generates an acoustic signal Xn that represents the waveform of the sound. That is, the sound collection system 13 generates N-channel acoustic signals X1 to XN in parallel in parallel with the progress of an event within the facility 200. The N-channel acoustic signals X1 to XN generated by the sound collection system 13 are transmitted to the control system 14 by wired communication or wireless communication. For convenience, an A / D converter that converts each acoustic signal Xn from analog to digital is omitted from the illustration.

[0018] Some of the N sound pickup units 15_1 to 15_N are installed in various equipment within the facility 200. Specifically, one or more sound pickup units 15_n are installed in each of the competition field where the event is actually held in the facility 200 and the spectator seats where the multiple spectators Ub are seated. For example, one or more sound pickup units 15_n are installed in each of the spectator seats for the spectators Ub who support the home team participating in the event and the spectator seats for the spectators Ub who support the away team. Therefore, sounds related to the event, such as cheers uttered by the spectators Ub, are picked up by the sound pickup unit 15_n. In addition, the other part of the N sound pickup units 15_1 to 15_N are attached to any of the multiple related parties Uc within the facility 200. For example, one or more sound pickup units 15_n are attached to the participants (players, coaches) or umpires of the event. One or more sound collection units 15_n may be attached to various objects (e.g., soccer balls or goal posts) used in the event. For example, the sound collection unit 15_n worn by a participant of the event collects sounds related to the event, such as the participant's vocalization, breathing, or movement sounds (e.g., the sound of kicking the ball).

[0019] As can be understood from the above description, N-channel sound signals X1-XN generated by collecting sounds at different positions in the facility 200 are generated in parallel with the progress of the event. That is, each sound collection unit 15_n generates a sound signal Xn for the event by collecting sounds at each position in the facility 200. Each sound signal Xn is collectively expressed as a signal representing a sound related to the event. The control system 14 is a computer system that causes each terminal device 22 to reproduce sounds corresponding to the N-channel sound signals X1-XN generated by the sound collection system 13. The control system 14 is capable of communicating with each of the multiple terminal devices 22 via the communication network 300.

[0020] 3 is a block diagram illustrating the configuration of each terminal device 22. The terminal device 22 includes a control device 41, a storage device 42, a communication device 43, an operation device 44, and a playback device 45. The terminal device 22 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0021] The control device 41 is composed of one or more processors that control each element of the terminal device 22. For example, the control device 41 is composed of one or more types of processors such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0022] The storage device 42 is one or more memories that store the programs executed by the control device 41 and various data used by the control device 41. The storage device 42 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 42 may be configured by combining a plurality of types of recording media. The communication device 43 communicates with the control system 14 via the communication network 300.

[0023] The operation device 44 is an input device that receives instructions from the user Ua. The operation device 44 is, for example, a plurality of operators operated by the user Ua, or a touch panel that detects contact by the user Ua.

[0024] The playback device 45 plays back video under the control of the control device 41. The playback device 45 includes a display device 46 that displays video and a sound emitting device 47 that plays back sound. Note that the playback device 45 may play back the content C that the communication device 43 receives from the distribution system 12. In other words, the playback device 21 may be omitted.

[0025] 4 is a block diagram illustrating an example of the configuration of the control system 14. The control system 14 includes a control device 51, a storage device 52, and a communication device 53. The control system 14 may be realized as a single device, or may be realized as a set of multiple devices configured separately from each other.

[0026] The control device 51 is composed of one or more processors that control each element of the control system 14. For example, the control device 51 is composed of one or more types of processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC.

[0027] The storage device 52 is one or more memories that store the programs executed by the control device 51 and various data used by the control device 51. The storage device 52 is configured by a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 52 may be configured by combining multiple types of recording media.

[0028] The communication device 53 communicates with each of the multiple terminal devices 22 via the communication network 300. The communication device 53 also communicates with the sound collection system 13 in the facility 200. Specifically, the communication device 53 receives N-channel acoustic signals X1 to XN generated by the sound collection system 13.

[0029] 5 is a block diagram illustrating an example of a functional configuration of the control system 14. The control device 51 executes a program stored in the storage device 52 to function as a plurality of elements (an information generating unit 61, a signal acquiring unit 62, an information acquiring unit 63, and a playback control unit 64).

[0030] The information generating unit 61 generates playback control information Q. The playback control information Q is data that specifies conditions (hereinafter referred to as “playback conditions”) related to the playback of each of the N-channel acoustic signals X1 to XN. The information generating unit 61 generates the playback control information Q for each of the multiple terminal devices 22.

[0031] 6 is a schematic diagram of playback control information Q corresponding to any one of the terminal devices 22. The playback control information Q in the first embodiment specifies whether or not to play back (enable / disable) each of the N-channel acoustic signals X1 to XN. In other words, the playback control information Q specifies whether or not each acoustic signal Xn is to be played back in the terminal device 22. As can be understood from the above explanation, the playback condition for each acoustic signal Xn in the first embodiment includes whether or not to play back the sound represented by the acoustic signal Xn.

[0032] The information generator 61 generates playback control information Q for each terminal device 22 in response to an instruction from a user Ua of the terminal device 22. The user Ua of the terminal device 22 operates the operation device 44 to instruct whether or not to play back each of the N-channel acoustic signals X1 to XN.

[0033] FIG. 7 is a schematic diagram of an operation screen G that the user Ua refers to in order to instruct whether or not to play each sound signal Xn. Before the start of the event, the operation screen G is displayed on the display device 46 of each terminal device 22. On the operation screen G, a sound collection position name G1 and an operation area G2 are displayed for each of the N-channel sound signals X1 to XN. The sound collection position name G1 of each sound signal Xn is the name of the location where the sound collection unit 15_n that generates the sound signal Xn is installed. The operation area G2 of each sound signal Xn is an operation image that allows the user Ua to instruct whether or not to enable (ON) / disable (OFF) the sound signal Xn. The user Ua refers to the operation screen G to instruct whether or not to play each sound signal Xn.

[0034] As illustrated in FIG. 5, instruction data Za representing an instruction by the user Ua is transmitted from the communication device 43 of the terminal device 22 to the control system 14. The instruction data Za is data representing the content (valid / invalid) of the instruction by the user Ua for each of the N-channel acoustic signals Xn. The information generation unit 61 of the control system 14 generates playback control information Q for each terminal device 22 in accordance with the instruction data Za received from that terminal device 22. The playback control information Q generated by the information generation unit 61 is stored in the storage device 52 for each terminal device 22. In addition, the information generation unit 61 changes the playback conditions of each acoustic signal Xn in the playback control information Q stored in the storage device 52 in accordance with the instruction (instruction data Za) from the user Ua of the terminal device 22.

[0035] 5 acquires N-channel acoustic signals X1 to XN from the sound collection system 13. Specifically, the signal acquisition unit 62 receives the N-channel acoustic signals X1 to XN transmitted from the sound collection system 13 by the communication device 53. The signal acquisition unit 62 acquires each acoustic signal Xn in parallel with the progress of the event. The information acquisition unit 63 acquires playback control information Q. Specifically, the information acquisition unit 63 reads out the playback control information Q of each terminal device 22 from the storage device 52.

[0036] The playback control unit 64 executes a process (hereinafter referred to as a "playback control process") Sa3 for causing each terminal device 22 to play back sounds (hereinafter referred to as "recorded sounds") corresponding to the N-channel sound signals X1 to XN acquired by the signal acquisition unit 62. The playback control process Sa3 is executed in parallel with the progress of the event. The playback control process Sa3 utilizes playback control information Q acquired by the information acquisition unit 63. Specifically, the playback control unit 64 causes each terminal device 22 to play back recorded sounds corresponding to one or more sound signals Xn of the N-channel sound signals X1 to XN under playback conditions specified by the playback control information Q of the terminal device 22.

[0037] The playback control unit 64 of the first embodiment generates a playback signal Y by performing audio processing Sc1 on N-channel audio signals X1 to XN. The playback signal Y is a signal that represents the waveform of the recorded sound. Specifically, the playback control unit 64 generates the playback signal Y by mixing one or more audio signals Xn that are set to "valid" in the playback control information Q among the N-channel audio signals X1 to XN. One or more audio signals Xn that are specified to be "invalid" in the playback control information Q among the N-channel audio signals X1 to XN are not used to generate the playback signal Y. Note that when only one audio signal Xn is set to "valid" among the N-channel audio signals X1 to XN, the audio signal Xn is adopted as the playback signal Y.

[0038] The playback control unit 64 transmits the playback signal Y generated by the acoustic processing Sc1 from the communication device 53 to the terminal device 22. The control device 41 of the terminal device 22 supplies the playback signal Y received from the control system 14 to the sound emitting device 47. Therefore, the recorded sound represented by the playback signal Y generated according to the playback control information Q of each terminal device 22 is reproduced from the sound emitting device 47. Since the playback control information Q is generated individually for each terminal device 22, the combination of the acoustic signals Xn constituting the recorded sound differs for each terminal device 22.

[0039] 8 is a flowchart illustrating a specific procedure of a process (hereinafter referred to as a "control process") Sa executed by the control device 51 of the control system 14. The control process Sa is started in response to an instruction from the administrator of the control system 14, and is executed in parallel with the progress of the event. The control process Sa is executed for each of the multiple terminal devices 22 for which the information generation unit 61 has generated playback control information Q.

[0040] When the control process Sa is started, the control device 51 (information acquisition unit 63) acquires playback control information Q from the storage device 52 (Sa1). The control device 51 (signal acquisition unit 62) acquires N-channel acoustic signals X1 to XN from the sound collection system 13 (Sa2). Then, the control device 51 (playback control unit 64) executes a playback control process Sa3 to which the playback control information Q and the N-channel acoustic signals X1 to XN are applied.

[0041] The control device 51 determines whether a predetermined end condition is met (Sa4). For example, when an instruction to end the control process Sa is received from the administrator, the control device 51 determines that the end condition is met. Note that the end condition may be, for example, the arrival of the time when the event ends. If the end condition is not met (Sa4: NO), the control device 51 shifts the process to step Sa2. That is, the acquisition of N-channel sound signals X1 to XN (Sa2) and the playback control process Sa3 are repeated. If the end condition is met (Sa4: YES), the control device 51 ends the control process Sa.

[0042] 9 is a flowchart illustrating a specific procedure of the playback control process Sa3 in the first embodiment. The playback control process Sa3 in the first embodiment includes an acoustic process Sc1 and a transmission process Sc2. In the acoustic process Sc1, the control device 51 generates a playback signal Y by mixing one or more acoustic signals Xn set to "valid" in the playback control information Q among the N-channel acoustic signals X1 to XN. The transmission process Sc2 is a process of transmitting the playback signal Y generated in the acoustic process Sc1 to the terminal device 22.

[0043] As described above, in the first embodiment, recorded sounds corresponding to one or more sound signals Xn among the N-channel sound signals X1 to XN generated by collecting sounds at different positions in the facility 200 are reproduced under the reproduction conditions specified by the reproduction control information Q. Therefore, the user Ua of the terminal device 22 can hear sounds at one or more positions among the multiple positions in the facility 200. In particular, in the first embodiment, the presence or absence of reproduction of each of the N-channel sound signals X1 to XN is controlled, so that the user Ua of each terminal device 22 can selectively hear sounds collected at one or more positions among the multiple positions in the facility 200. For example, the user Ua can selectively hear sounds collected by the sound collection unit 15_n attached to the relevant person Uc such as a participant or a referee. In addition, the user Ua can selectively hear sounds collected by the sound collection unit 15_n installed in a specific spectator seat (for example, the home team's spectator seat) in the facility 200.

[0044] Furthermore, in the first embodiment, the playback control information Q of the terminal device 22 is generated in response to an instruction from the user Ua of the terminal device 22. Therefore, the playback of recorded sounds according to the preferences of each user Ua is realized.

[0045] B: Second embodiment A second embodiment will be described. Note that, for elements having the same functions as those in the first embodiment in each of the following exemplary aspects, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0046] 10 is a block diagram illustrating a functional configuration of a control system 14 in the second embodiment. An information acquisition unit 63 in the second embodiment selectively acquires, for each terminal device 22, one of a plurality of pieces of playback control information Q (Q1-Q3) having different playback conditions for each audio signal Xn. Each of the plurality of pieces of playback control information Q has different playback conditions (enabled / disabled) for each audio signal Xn.

[0047] Specifically, the playback control information Q1 is generated individually for each terminal device 22 in response to an instruction from the user Ua of the terminal device 22, as in the example of the first embodiment. On the other hand, the playback control information Q2 and the playback control information Q3 are preset data that are prepared in advance regardless of an instruction from the user Ua. While the playback control information Q1 is information that is generated for each terminal device 22, the playback control information Q2 and the playback control information Q3 are shared by multiple terminal devices 22.

[0048] In the reproduction control information Q2, among the N-channel sound signals X1 to XN, one or more sound signals Xn corresponding to the sound pickup units 15_n attached to specific related persons Uc such as contestants are set to "valid". On the other hand, in the reproduction control information Q3, among the N-channel sound signals X1 to XN, one or more sound signals Xn corresponding to the sound pickup units 15_n installed in facilities such as spectator seats in the facility 200 are set to "valid". For example, (1) reproduction control information Q in which the sound signal Xn of the sound pickup units 15_n installed in a first range (e.g., the home team's spectator seats) of the spectator seats in the facility 200 is set to "valid", and (2) reproduction control information Q in which the sound signal Xn of the sound pickup units 15_n installed in a second range (e.g., the away team's spectator seats) different from the first range is set to "valid" may be stored in the storage device 52.

[0049] A user Ua of each terminal device 22 can select one of the multiple pieces of playback control information Q by operating the operation device 44. The user Ua can select the desired playback control information Q at any time during the period during which the event is held (hereinafter referred to as the "implementation period"). Selection data Zb specifying the playback control information Q selected by the user Ua is transmitted from the communication device 43 to the control system 14. The communication device 43 of the control system 14 receives the selection data Zb. The information acquisition unit 63 of FIG. 10 acquires from the storage device 52 the playback control information Q specified by the selection data Zb received from the terminal device 22 among the multiple pieces of playback control information Q.

[0050] 11 is a flowchart illustrating a specific procedure of the control process Sa in the second embodiment. The control process Sa is started in response to an instruction from an administrator of the control system 14, and is executed in parallel with the progress of the event. The control process Sa is executed for each of the multiple terminal devices 22.

[0051] When the control process Sa is started, the control device 51 (information acquisition unit 63) acquires the playback control information Q from the storage device 52 (Sa1). Immediately after the control process Sa is started, a specific piece of playback control information Q (e.g., playback control information Q1) is acquired from the multiple pieces of playback control information Q. The control device 51 (signal acquisition unit 62) acquires N-channel acoustic signals X1 to XN from the sound collection system 13 (Sa2). Then, the control device 51 (playback control unit 64) executes a playback control process Sa3 to which the playback control information Q and the N-channel acoustic signals X1 to XN are applied. The content of the playback control process Sa3 is the same as that of the first embodiment. That is, a playback signal Y is generated from one or more acoustic signals Xn that are set to "valid" in the playback control information Q acquired by the information acquisition unit 63, among the N-channel acoustic signals X1 to XN acquired by the signal acquisition unit 62.

[0052] The control device 51 judges whether a predetermined end condition is satisfied (Sa4). When the end condition is satisfied (Sa4: YES), the control device 51 terminates the control process Sa. On the other hand, when the end condition is not satisfied (Sa4: NO), the control device 51 (information acquisition unit 63) judges whether to change the playback control information Q (Sa5). For example, when the communication device 53 receives the selection data Zb from the terminal device 22, the control device 51 judges that the playback control information Q is changed. When the playback control information Q is changed (Sa5: YES), the control device 51 shifts the process to step Sa1. In step Sa1 after the shift, the control device 51 acquires the playback control information Q designated by the selection data Zb from the storage device 52. That is, the playback control information Q applied to the playback control process Sa3 is changed. On the other hand, when the playback control information Q is not changed (Sa5: NO), the control device 51 shifts the process to step Sa2. That is, the playback control information Q is not changed.

[0053] As can be understood from the above description, in the second embodiment, the playback control information Q changes over time within the implementation period. That is, the playback conditions of each sound signal Xn change over time. Therefore, for example, as illustrated in FIG. 12, during a period Ta within the implementation period, playback control information Qa (Q1-Q3) is applied to the playback control process Sa3, and during a period Tb different from the period Ta, playback control information Qb (Q1-Q3, Qb≠Qa) different from the playback control information Qa is applied to the playback control process Sa3. That is, the information acquisition unit 63 selects the playback control information Qa during the period Ta, and acquires the playback control information Qb during the period Tb. The period Ta is an example of the "first period", and the period Tb is an example of the "second period". Moreover, the playback control information Qa is an example of the "first playback control information", and the playback control information Qb is an example of the "second playback control information".

[0054] The second embodiment also achieves the same effect as the first embodiment. Moreover, in the second embodiment, since any one of the multiple pieces of playback control information Q is selectively applied to the playback control process Sa3, it is possible to diversify the sounds played by the sound emitting device 47, compared to a configuration in which only one piece of playback control information Q is used. Furthermore, since the playback control information Q applied to the playback control process Sa3 changes over time, it is possible to realize a variety of playback in which the playback conditions of the sounds represented by each sound signal Xn change over time. Since the playback control information Q2 and the playback control information Q3 are information prepared in advance, there is also an advantage that the load on the user Ua or the load on the information generating unit 61 is reduced, compared to a configuration in which multiple pieces of playback control information Q are generated in response to an instruction from the user Ua.

[0055] C: Third embodiment 13 is a block diagram illustrating a functional configuration of the control system 14 in the third embodiment. The control device 51 in the third embodiment functions as a situation analyzer 65 in addition to the same elements as in the first embodiment (information generator 61, signal acquirer 62, information acquirer 63, and playback controller 64). As in the second embodiment, the information acquirer 63 in the third embodiment selectively acquires, for each terminal device 22, one of a plurality of pieces of playback control information Q (Q1-Q3) having different playback conditions for each sound signal Xn. The operations of the other elements (information generator 61, signal acquirer 62, and playback controller 64) are the same as in the first embodiment.

[0056] The situation analysis unit 65 analyzes the progress of the event (hereinafter referred to as the "progress status"). Specifically, the situation analysis unit 65 estimates the progress status by analyzing the N-channel audio signals X1 to XN acquired by the signal acquisition unit 62. For example, when the volume of the audio signal Xn increases suddenly, the situation analysis unit 65 estimates that the progress status is that one of the contestants has gained a point. Note that the information used to estimate the progress status is not limited to the audio signal Xn. For example, the situation analysis unit 65 may analyze the progress status by analyzing the video recorded by the recording system 11. For example, the situation analysis unit 65 estimates each scene of the video as the progress status by using a known scene analysis technique.

[0057] The user Ua can transmit desired posted information from the terminal device 22 using a social networking service (SNS). The user Ua creates and transmits the posted information while viewing the content C reproduced by the reproduction device 21. The posted information includes character strings related to the progress of the event, such as impressions on viewing the content C. The situation analysis unit 65 may estimate the progress of the event by referring to the posted information transmitted from the terminal device 22 of each user Ua.

[0058] The control device 51 of the third embodiment executes the control process Sa similar to that of the second embodiment. However, the information acquisition unit 63 of the third embodiment acquires the playback control information Q corresponding to the progress status estimated by the situation analysis unit 65 (Sa1). For example, the information acquisition unit 63 identifies the playback control information Q corresponding to the progress status estimated by the situation analysis unit 65 by referring to a table in which the playback control information Q is registered for each progress status. The playback control information Q acquired by the information acquisition unit 63 is applied to the playback control process Sa3 in the same manner as in the first or second embodiment. Also, in step Sa5 of the control process Sa, the information acquisition unit 63 determines that the playback control information Q is to be changed when the progress status estimated by the situation analysis unit 65 has changed (Sa5: YES). That is, the playback control information Q is changed every time the progress status changes.

[0059] The third embodiment also achieves the same effects as the second embodiment. Moreover, in the third embodiment, the playback control information Q is changed according to the progress of the event, so that sounds corresponding to each sound signal Xn can be played back under appropriate playback conditions according to the progress of the event.

[0060] D: Fourth embodiment 14 is a schematic diagram of playback control information Q in the fourth embodiment. The playback control information Q in the fourth embodiment specifies a playback volume V (V1-VN) for each of N-channel audio signals X1-XN. That is, the playback condition for each audio signal Xn in the fourth embodiment includes a playback volume Vn of the sound represented by the audio signal Xn.

[0061] The user Ua operates the operation device 44 to specify the playback volume Vn for each of the N-channel acoustic signals X1 to XN. The user Ua sets the playback volume Vn of the acoustic signal Xn that the user wants to preferentially listen to among the N channels to a large value, and sets the playback volume Vn of the acoustic signal Xn that is not important to the user Ua to a small value. The playback volume Vn of sounds that the user Ua does not need to listen to is set to zero. The information generation unit 61 generates playback control information Q for each terminal device 22 in response to an instruction (instruction data Za) from the user Ua of that terminal device 22.

[0062] In the acoustic processing Sc1 of the fourth embodiment, the playback control unit 64 generates a playback signal Y by mixing the N-channel acoustic signals X1 to XN according to the playback conditions specified by the playback control information Q. Specifically, the playback control unit 64 generates the playback signal Y by adding the N-channel acoustic signals X1 to XN using the playback volume Vn specified by the playback control information Q as a weighted value (i.e., weighted sum). The acoustic processing Sc1 of the first embodiment can also be said as a form in which the playback volume Vn is set to a binary value (0 / 1) in the acoustic processing Sc1 of the fourth embodiment. As can be understood from the above description, the playback control processing Sa3 in the first and fourth embodiments includes the acoustic processing Sc1 that generates the playback signal Y by mixing the N-channel acoustic signals X1 to XN according to the playback conditions specified by the playback control information Q. The operation in which the control system 14 transmits the playback signal Y to the terminal device 22 and the sound emitting device 47 plays back the recorded sound is the same as in the first embodiment.

[0063] The fourth embodiment also achieves the same effects as the first embodiment. Moreover, in the fourth embodiment, the playback volume Vn is controlled for each of the N-channel acoustic signals X1 to XN. Therefore, the user Ua of each terminal device 22 can preferentially listen to sounds collected at one or more positions among the multiple positions in the facility 200.

[0064] The configuration of the second or third embodiment in which the playback control information Q changes over time is similarly applied to the fourth embodiment in which the playback control information Q specifies the playback volume Vn of each audio signal Xn. For example, the information acquisition unit 63 selectively acquires one of a plurality of pieces of playback control information Q in which the playback volume Vn of each audio signal Xn differs.

[0065] E: Fifth embodiment 15 is a block diagram illustrating the configuration of a sound emitting device 47 of each terminal device 22 in the fifth embodiment. The sound emitting device 47 in the fifth embodiment includes N sound emitting units 48. Specifically, the sound emitting device 47 in the fifth embodiment is an N-channel surround system configured separately from the terminal device 22. The N sound emitting units 48 are installed around the user Ua.

[0066] 16 is a schematic diagram of the reproduction control information Q in the fifth embodiment. The reproduction control information Q in the fifth embodiment specifies the identification information (#1, #2, ...) of the sound emitting unit 48 to which each of the N-channel sound signals X1 to XN is supplied. That is, the reproduction control information Q specifies one of the N sound emitting units 48 of the sound emitting device 47 as the supply destination of each sound signal Xn. As can be understood from the above description, the reproduction condition of each sound signal Xn in the fifth embodiment includes specification of the sound emitting unit 48 (i.e., output channel) to which the sound signal Xn is supplied.

[0067] The user Ua operates the operation device 44 to specify which of the N sound emitting units 48 is to be used for each of the N channel sound signals X1 to XN. That is, each sound signal Xn is assigned to one of the N sound emitting units 48. The information generation unit 61 generates playback control information Q for each terminal device 22 in response to an instruction (instruction data Za) from the user Ua of that terminal device 22.

[0068] In the reproduction control process Sa3, the reproduction control unit 64 of the fifth embodiment designates the sound emitting unit 48 designated by the reproduction control information Q for each sound signal Xn as a supply destination of the sound signal Xn, and transmits the N-channel sound signals X1 to XN from the communication device 53 to the terminal device 22. The control device 41 of the terminal device 22 supplies each sound signal Xn received from the control system 14 to the sound emitting unit 48 designated as a supply destination for the sound signal Xn among the N sound emitting units 48 of the sound emitting device 47. Therefore, a sound corresponding to each sound signal Xn is reproduced from the sound emitting unit 48 designated by the reproduction control information Q among the N sound emitting units 48. Note that in the fifth embodiment, the sound process Sc1 for mixing the sound signals Xn is not executed.

[0069] The fifth embodiment also achieves the same effect as the first embodiment. In the fifth embodiment, a combination of each of the N-channel acoustic signals Xn and the sound emitting unit 48 that reproduces the sound corresponding to the acoustic signal Xn is set for each terminal device 22 according to the reproduction control information Q. Therefore, the sound at one or more positions among the multiple positions in the facility 200 can be selectively or preferentially heard by each of the N sound emitting units 48. For example, a viewing environment is realized in which the reproduced sound of the acoustic signal Xn corresponding to the sound collecting unit 15_n attached to the related person Uc in the facility 200 is emitted from the sound emitting unit 48 installed in front of the user Ua of the sound emitting device 47, and the reproduced sound of the acoustic signal Xn corresponding to the sound collecting unit 15_n installed in the spectator seats in the facility 200 is emitted from the sound emitting unit 48 installed behind the user Ua of the sound emitting device 47.

[0070] In the above description, a sound represented by one acoustic signal Xn is reproduced by one sound emitting unit 48, but a sound represented by a signal generated by mixing two or more acoustic signals Xn may be reproduced by one sound emitting unit 48. A signal generated by the above-mentioned acoustic processing Sc1 for two or more acoustic signals Xn may be supplied to one sound emitting unit 48. That is, the configuration of the fourth embodiment is also applied to the fifth embodiment. As can be understood from the above description, the correspondence between the acoustic signal Xn and the sound emitting unit 48 is not limited to a one-to-one relationship.

[0071] The configuration of the second or third embodiment in which the playback control information Q changes over time is also applied to the fifth embodiment in which the playback control information Q specifies the supply destination of each audio signal Xn. For example, the information acquisition unit 63 selectively acquires one of a plurality of pieces of playback control information Q each of which has a different supply destination for each audio signal Xn.

[0072] F: Sixth embodiment 17 is an explanatory diagram of the operation of the playback control unit 64 in the sixth embodiment. Each sound collection unit 15_n of the sound collection system 13 in the sixth embodiment includes a clock circuit (not shown) that measures the current time. The sound collection unit 15_n adds time information τ (τ1, τ2, ...) to the sound signal Xn generated by collecting sound. The time information τ is data representing the time of sound collection. For example, at a time that arrives at a predetermined period, the time information τ representing that time is added to the sound signal Xn.

[0073] The sound collection performance of each sound collection unit 15_n of the sound collection system 13 or the communication environment in which each sound signal Xn is transmitted differs among the N sound collection units 15_1 to 15_N. Therefore, the delay amount of the sound signal Xn generated by the sound collection unit 15_n or the delay amount of the transmission of the sound signal Xn differs for each sound signal Xn. Due to the above differences, there are cases where the positions on the time axis are not consistent among the N-channel sound signals X1 to XN acquired by the signal acquisition unit 62 of the control system 14.

[0074] For example, let us focus on acoustic signals Xn1 and Xn2 of N-channel acoustic signals X1 to XN for convenience (n1, n2 = 1 to N, n1 ≠ n2). As illustrated in FIG. 17, a point in time in acoustic signal Xn1 corresponding to a specific time does not completely match a point in time in acoustic signal Xn2 corresponding to the same time on the time axis. Specifically, acoustic signal Xn1 arrives at control system 14 in a delayed state compared to acoustic signal Xn2. In consideration of the above circumstances, in the sixth embodiment, the positions on the time axis of N-channel acoustic signals X1 to XN are matched with each other.

[0075] 18 is a flowchart illustrating a specific procedure of the playback control process Sa3 in the sixth embodiment. The playback control process Sa3 in the sixth embodiment includes a synchronization process Sc3 in addition to the acoustic process Sc1 and the transmission process Sc2 similar to those in the first embodiment. The synchronization process Sc3 is executed before the start of the acoustic process Sc1.

[0076] The synchronization process Sc3 is a signal process for matching the positions on the time axis among the N-channel acoustic signals X1 to XN acquired by the signal acquisition unit 62. Specifically, as illustrated in Fig. 17, the playback control unit 64 adjusts the positions on the time axis of each of the acoustic signals Xn1 and Xn2 so that the time point indicated by each piece of time information τ in the acoustic signal Xn1 coincides with the time point indicated by each piece of time information τ in the acoustic signal Xn2 on the time axis. Fig. 17 illustrates an example in which the position on the time axis of the acoustic signal Xn2 is adjusted based on the acoustic signal Xn1. As can be understood from the above explanation, the synchronization process Sc3 is a process for reducing the time difference between the N-channel acoustic signals X1 to XN.

[0077] The playback control unit 64 executes the acoustic processing Sc1 on the N-channel acoustic signals X1 to XN after the synchronization processing Sc3 has been executed. The specific processing procedures of the acoustic processing Sc1 and the transmission processing Sc2 are similar to those in the first embodiment.

[0078] The sixth embodiment also achieves the same effects as the first embodiment. In the sixth embodiment, the positions on the time axis are aligned among the N-channel acoustic signals X1 to XN. Therefore, the user Ua can hear a natural recorded sound with a reduced time difference between each acoustic signal Xn. The configuration of the sixth embodiment is also applicable to the second to fifth embodiments.

[0079] G: Seventh embodiment In the first to sixth embodiments, a configuration has been illustrated in which the control process Sa is executed in the control system 14. In the seventh embodiment, a similar control process Sb is executed in each terminal device 22.

[0080] 19 is a block diagram of an information system 100 and a terminal device 22 in the seventh embodiment. A control system 14 in the seventh embodiment transmits N-channel acoustic signals X1-XN generated by a sound collection system 13 to each of a plurality of terminal devices 22. A communication device 43 of each terminal device 22 receives the N-channel acoustic signals X1-XN transmitted from the control system 14 via a communication network 300. That is, the control system 14 functions as a device that relays each acoustic signal Xn.

[0081] 20 is a block diagram illustrating a functional configuration of the terminal device 22 in the seventh embodiment. The control device 41 of the terminal device 22 executes a program stored in the storage device 42 to function as a plurality of elements (an information generating unit 71, a signal acquiring unit 72, an information acquiring unit 73, and a playback control unit 74).

[0082] The information generator 71 generates playback control information Q, similar to the information generator 61 in the first embodiment. The playback control information Q specifies playback conditions for each of the N-channel acoustic signals X1 to XN. As in the first embodiment, the playback condition for each acoustic signal Xn in the seventh embodiment is whether or not to play back the sound represented by the acoustic signal Xn.

[0083] As in the first embodiment, the user Ua instructs whether or not to play each sound signal Xn while referring to the operation screen G (FIG. 7) displayed on the display device 46. The information generation unit 71 generates playback control information Q in response to an operation on the operation device 44. The playback control information Q generated by the information generation unit 71 is stored in the storage device 42. In addition, the information generation unit 71 changes the playback conditions of each sound signal Xn in the playback control information Q stored in the storage device 42 in response to an instruction from the user Ua to the operation device 44.

[0084] 20 acquires N-channel acoustic signals X1 to XN. Specifically, the signal acquirer 72 receives the N-channel acoustic signals X1 to XN transmitted from the control system 14 by the communication device 43. The signal acquirer 72 acquires each acoustic signal Xn in parallel with the progress of the event. The information acquirer 73 acquires playback control information Q. Specifically, the information acquirer 73 reads out the playback control information Q from the storage device 42.

[0085] The playback control unit 74 executes a playback control process Sb3 for causing the playback device 45 (sound emitting device 47) to play back recorded sounds corresponding to the N-channel sound signals X1-XN acquired by the signal acquisition unit 62. The playback control process Sb3 is executed in parallel with the progress of the event. The playback control unit 74 of the seventh embodiment causes the sound emitting device 47 to play back recorded sounds corresponding to one or more sound signals Xn of the N-channel sound signals X1-XN under playback conditions specified by the playback control information Q.

[0086] 21 is a flowchart illustrating a specific procedure of the control process Sb executed by the control device 41 of the terminal device 22. The control process Sb is started in response to an instruction from the user Ua to the operation device 44, and is executed in parallel with the progress of the event.

[0087] When the control process Sb is started, the control device 41 (information acquisition unit 73) acquires the playback control information Q from the storage device 42 (Sb1). The control device 41 (signal acquisition unit 72) acquires the N-channel acoustic signals X1 to XN from the control system 14 (Sb2). Then, the control device 41 (playback control unit 74) executes the playback control process Sb3 to which the playback control information Q and the N-channel acoustic signals X1 to XN are applied. The playback control process Sb3 includes the same acoustic process Sc1 as in the first embodiment. The acoustic process Sc1 is a signal process that generates a playback signal Y by mixing one or more acoustic signals Xn set to "valid" in the playback control information Q among the N-channel acoustic signals X1 to XN. The playback signal Y is supplied to the sound emitting device 47 to play back the recorded sound.

[0088] The control device 41 determines whether a predetermined end condition is satisfied (Sb4) as in the first embodiment. If the end condition is not satisfied (Sb4: NO), the control device 41 shifts the process to step Sb2 and repeats the acquisition of each sound signal Xn (Sb2) and the playback control process Sb3. On the other hand, if the end condition is satisfied (Sb4: YES), the control device 41 ends the control process Sb.

[0089] As described above, in the seventh embodiment, recorded sounds corresponding to one or more acoustic signals Xn among N-channel acoustic signals X1 to XN generated by collecting sounds at different positions in the facility 200 are reproduced under the reproduction conditions specified by the reproduction control information Q. Therefore, similar to the first embodiment, the user Ua of the terminal device 22 can hear sounds at one or more positions among the multiple positions in the facility 200.

[0090] The second to sixth embodiments are similarly applied to the seventh embodiment. That is, the configurations and operations exemplified for the control system 14 in the second to sixth embodiments are similarly applied to the terminal device 22 in the seventh embodiment.

[0091] For example, in a case where the second embodiment is applied to the seventh embodiment, the control device 41 (information acquisition unit 73) selects one of the multiple pieces of playback control information Q stored in the storage device 42. That is, the playback control information Q applied to the playback control process Sb3 changes over time. Also, in a case where the third embodiment is applied to the seventh embodiment, the control device 41 (information acquisition unit 73) acquires, from the multiple pieces of playback control information Q, the playback control information Q that corresponds to the progress of the event.

[0092] In a configuration in which the fourth embodiment is applied to the seventh embodiment, the control device 41 (playback control unit 74) executes an audio process Sc1 that mixes each audio signal Xn at a playback volume Vn specified by playback control information Q. In a configuration in which the fifth embodiment is applied to the seventh embodiment, the control device 41 (playback control unit 74) supplies each audio signal Xn to a supply destination specified by playback control information Q among the N sound emitting units 48 of the sound emitting device 47. As in the sixth embodiment, the playback control process Sb3 executed by the control device 41 of the terminal device 22 may include a synchronization process Sc3 that aligns the positions on the time axis among the audio signals X1 to XN of the N channels.

[0093] H: Variation Specific modified embodiments that can be added to each of the above-mentioned embodiments are illustrated below. A plurality of embodiments arbitrarily selected from the following examples may be appropriately combined as long as they are not mutually contradictory.

[0094] (1) In the second embodiment, a form in which one of the multiple pieces of playback control information Q is selected in response to an instruction from the user Ua has been exemplified, but a specific method for the information acquisition unit 63 to acquire one of the multiple pieces of playback control information Q is not limited to the above example. For example, the information acquisition unit 63 may select one of the multiple pieces of playback control information Q in response to information (hereinafter referred to as "registered information") previously registered in the storage device 52 of the control system 14 for each user Ua. The registered information includes, for example, a designation of a team that the user Ua supports. The information acquisition unit 63 selects playback control information Q in which the sound signal Xn of the sound collection unit 15_n within a range of the spectator seats in the facility 200 corresponding to the team designated by the registered information is set to "valid." According to the above configuration, the user Ua can hear the sound collected within a range of the spectator seats corresponding to the team that the user Ua supports (for example, the spectator seats of the home team). Note that the above example focuses on the information acquisition unit 63, but a similar form is also adopted for the information acquisition unit 73 of the terminal device 22.

[0095] (2) In each of the above-mentioned embodiments, the reproduction of the content C by the reproduction device 21 and the reproduction of the sound by the sound emitting device 47 are performed in parallel. Since the distribution of the content C is delayed due to, for example, the communication environment, the content C and the recorded sound may not match on the time axis. In consideration of the above circumstances, a form in which the position of each sound signal Xn on the time axis is adjusted so that the content C and each sound signal Xn (or the reproduction signal Y) match on the time axis is also assumed. For example, the reproduction control unit 64 adjusts the position of the sound signal Xn on the time axis so that the time indicated by the time information of the video data C1 of the content C matches the time indicated by the time information τ of each sound signal Xn on the time axis. According to the above-mentioned configuration, the reproduction device 45 can reproduce a sound that is time-matched with the video represented by the video data C1. Note that, if the distribution delay of the content C can be predicted in advance, the reproduction control unit 64 may delay each sound signal Xn by a time equivalent to the distribution delay. In other words, the comparison between the video data C1 and the sound signal Xn is omitted. Although the above description focuses on the playback control unit 64, the playback control unit 74 of the terminal device 22 may execute the same adjustment.

[0096] (3) The user Ua who can use the service using the control system 14 may be restricted. Specifically, a form in which the service is provided only to the user Ua who is viewing the content C is assumed. For example, permission data representing permission to use is superimposed on the sound data C2 of the content C. The permission data is added to, for example, a band above the audible band of the sound represented by the sound data C2. The control device 41 of the terminal device 22 extracts the permission data by collecting the sound reproduced by the sound emitting device 24 of the playback device 21. That is, the permission data is transmitted to the periphery of the playback device 21 by acoustic communication using the sound emitting device 24 as a transmitter. The communication device 43 transmits the permission data extracted by the control device 41 to the control system 14. The control system 14 provides the service according to each of the above forms only to the terminal device 22 that transmitted the permission data. According to the above configuration, the service can be provided limitedly to the user Ua who is actually viewing the content C.

[0097] (4) Not only the sound that is the original target of sound collection (hereinafter referred to as "target sound") but also non-target sound reaches each sound collection unit 15_n of the sound collection system 13. Non-target sound is also called spill sound. For example, not only the target sound generated by the relevant person Uc but also non-target sound around the relevant person Uc reaches the sound collection unit 15_n attached to the relevant person Uc. Also, not only the target sound generated by the spectator Ub in the first range but also non-target sound generated by the spectator Ub in a second range different from the first range reaches the sound collection unit 15_n installed in a first range of the spectator seats. The reproduction control unit 64 may perform signal processing for reducing non-target sound on each sound signal Xn. For example, the reproduction control unit 64 performs various signal processing (e.g., sound source separation technology) for emphasizing the target sound in each sound signal Xn. According to the above configuration, the user Ua can clearly hear the target sound picked up by the specific sound pickup unit 15_n in the facility 200. Note that, although the above description focuses on the playback control unit 64, the playback control unit 74 of the terminal device 22 may execute the same process.

[0098] (5) In each of the above-mentioned embodiments, the recorded sound represented by the playback signal Y is reproduced by the sound emitting device 47, but the character string represented by the playback signal Y (hereinafter referred to as "recorded character string") may be displayed on the display device 46. For example, the control device 41 of the terminal device 22 estimates the recorded character string by speech recognition processing on the playback signal Y, and causes the display device 46 to display the recorded character string. The recorded character string expressed in a first language may be translated into a second language by a known machine translation processing and then displayed on the display device 46. In the above description, attention has been paid to the playback signal Y generated by the acoustic processing Sc1, but a similar configuration is also adopted in the fifth embodiment in which the acoustic processing Sc1 is not executed. For example, the control device 41 of the terminal device 22 estimates the recorded character string by speech recognition processing on one or more acoustic signals Xn of the N-channel acoustic signals X1 to XN supplied from the control system 14, and causes the display device 46 to display the recorded character string.

[0099] (6) Focus on the sound collection unit 15_n attached to the person Uc among the N sound collection units 15_1 to 15_N. The control device 51 of the control system 14 may estimate the person Uc (e.g., a specific contestant) by speaker identification processing for the acoustic signal Xn generated by the sound collection unit 15_n, and provide the terminal device 22 with related information related to the person Uc. For example, among a plurality of related information previously stored in the storage device 52 for different people Uc, related information corresponding to the person Uc estimated by the speaker identification processing is transmitted from the control system 14 to the terminal device 22. The related information includes various information such as the name, face photo, or profile of the person Uc. The control device 41 of the terminal device 22 causes the display device 46 to display the related information received from the control system 14. According to the above configuration, the user Ua can check the related information of the person Uc while listening to the voice uttered by the person Uc.

[0100] In the above description, the speaker is estimated by speaker identification processing for the acoustic signal Xn, but the speaker may be identified by using identification information set for each sound collection unit 15_n. For example, the sound collection unit 15_n transmits the identification information of the sound collection unit 15_n together with the acoustic signal Xn to the control system 14. The control device 51 provides the terminal device 22 with related information corresponding to the identification information among the multiple related information stored in the storage device 52. In the above configuration, speaker identification information for the acoustic signal Xn is not necessary. In the above description, the control device 51 of the control system 14 identifies the related information, but the control device 41 of the terminal device 22 may identify the related information by a similar process.

[0101] (7) In the above-mentioned embodiments, N sound collection units 15_1 to 15_N installed in the facility 200 where the event is held are used, but the location where each sound collection unit 15_n that generates the sound signal Xn is installed is not limited to within the facility 200. For example, a form in which each user Ua can use the sound collection unit 15_n is also assumed. Specifically, a sound collection device (microphone) mounted on the terminal device 22 may be used as the sound collection unit 15_n to generate the sound signal Xn. Each user Ua utters a sound related to the event while viewing the content C reproduced by the reproduction device 45. For example, each user Ua utters a sound that reports on the content C or a sound that expresses the user Ua's impressions of the content C.

[0102] Each sound collection unit 15_n collects sounds related to the above-mentioned events, thereby generating an audio signal Xn in parallel with the progress of the event. The signal acquisition unit 62 acquires N-channel audio signals X1 to XN including the audio signal Xn generated by the sound collection unit 15_n in the facility 200 and the audio signal Xn representing the sound uttered by the user Ua. The operation of the information acquisition unit 63 to acquire the playback control information Q and the operation of the playback control unit 64 to execute the playback control process Sa3 applying the playback control information Q are the same as those in the above-mentioned embodiments.

[0103] According to the above embodiment, it is possible to cause each terminal device 22 to play back recorded sounds including the voice of the user Ua who personally speaks about the event. Specifically, it is possible for each terminal device 22 to play back recorded sounds that are a combination of one or more voices selected from among a plurality of voices including voices collected in the facility 200 and voices of various users Ua related to the sports of the event, according to the playback control information Q. For example, recorded sounds that combine various voices, such as the voice of a user Ua whose hobby is playing or watching a sport related to the event, or the voice of a student user Ua who is involved in a club activity for the sport, are played back by the terminal device 22. It is also possible to play back recorded sounds including the voice of a user Ua who is viewing content C in a first country, by a terminal device 22 in a second country different from the first country.

[0104] In the above description, the recorded sound includes a voice picked up inside the facility 200, but a configuration is also envisaged in which the recorded sound includes only a voice picked up outside the facility 200 (e.g., the voice of the user Ua). Similar modified examples are also applied to the second to sixth embodiments. In the above description, the control system 14 is assumed to execute the control process Sa, but similar modified examples are also applied to the seventh embodiment in which the terminal device 22 executes the control process Sb.

[0105] As can be understood from the above examples, the signal acquisition unit (62, 72) is comprehensively expressed as an element that acquires a plurality of sound signals Xn generated for an event by collecting sounds at different positions. A typical example of the "different positions" is a position in the facility 200 where the event is held, as in the examples of each of the above-mentioned embodiments, but as in the example of this modified example, any position outside the facility 200 (e.g., the position of the user Ua) may also be included in the "different positions." In addition, the "sound signal Xn generated for the event" refers to a signal that represents any sound related to the event in terms of position or content, such as a sound generated in the facility 200 where the event is held (e.g., the voice of a spectator Ub or a related person Uc), or a voice explaining or reporting the event.

[0106] (8) In each of the above-described embodiments, the playback signal Y is supplied to the sound emitting device 47 to reproduce the recorded sound, but the sound emitting device 24 of the playback device 21 may be used to reproduce the recorded sound. For example, the control device 41 of the terminal device 22 transmits the playback signal Y received from the control system 14 to the playback device 21. The sound emitting device 24 of the playback device 21 reproduces a mixed sound of the sound represented by the acoustic data C2 of the content C and the recorded sound represented by the playback signal Y.

[0107] (9) Although sporting events are given as examples of the above-mentioned forms, the events to which the above-mentioned forms apply are not limited to these examples. For example, the above-mentioned forms apply to various events held for a specific purpose, such as music events such as live performances or concerts where music is played, dance events where dancers perform, theater events where actors perform, lecture events where speakers give lectures, and educational events where various educational institutions such as schools and cram schools provide lessons to students.

[0108] (10) As described above, the functions of the control system 14 exemplified above are realized by the cooperation of one or more processors constituting the control device 51 and the program stored in the storage device 52. Similarly, the functions of the terminal device 22 are realized by the cooperation of one or more processors constituting the control device 41 and the program stored in the storage device 42.

[0109] The program may be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and is exemplified by an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that the non-transitory recording medium includes any recording medium other than a transitory, propagating signal, and does not exclude volatile recording media. In addition, in a configuration in which a distribution device distributes a program via a communication network, the recording medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.

[0110] I: Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0111] A playback control method according to one aspect (aspect 1) of the present disclosure includes acquiring a plurality of sound signals generated for an event by sound collection at different positions in parallel with the progress of the event, acquiring playback control information for each of a plurality of terminal devices that specifies playback conditions for each of the plurality of sound signals, executing a playback control process for each of the plurality of terminal devices, and causing a playback device to play sounds corresponding to one or more of the plurality of sound signals in parallel with the progress of the event under the playback conditions specified by the playback control information of the terminal device in the playback control process. In the above configuration, sounds corresponding to one or more of the plurality of sound signals generated for the event by sound collection at different positions are played under the playback conditions specified by the playback control information. Therefore, a user of a terminal device can selectively or preferentially listen to sounds collected at one or more of the plurality of positions.

[0112] A playback control method according to another aspect (aspect 2) of the present disclosure includes a terminal device acquiring a plurality of sound signals generated for an event by sound collection at different positions in parallel with the progress of the event, acquiring playback control information specifying playback conditions for each of the plurality of sound signals, and executing a playback control process, in which the playback device is caused to play back sounds corresponding to one or more of the plurality of sound signals in parallel with the progress of the event under the playback conditions specified by the playback control information. In the above configuration, sounds corresponding to one or more of the plurality of sound signals generated for the event by sound collection at different positions are played back under the playback conditions specified by the playback control information. Thus, a user of the terminal device can selectively or preferentially listen to sounds collected at one or more of the plurality of positions.

[0113] In a specific example (Aspect 3) of Aspect 1 or Aspect 2, the different positions are positions present in a location where the event is held. In the above aspect, sounds corresponding to one or more of a plurality of sound signals generated by collecting sounds at different positions in the location where the event is held are reproduced.

[0114] In a specific example (Aspect 4) of any one of Aspects 1 to 3, the reproduction condition of each of the plurality of sound signals includes whether or not to reproduce the sound represented by the sound signal. In the above configuration, whether or not to reproduce each of the plurality of sound signals is controlled. Therefore, a user of the terminal device can selectively listen to sounds at one or more positions among the plurality of positions.

[0115] In a specific example (Aspect 5) of any one of Aspects 1 to 4, the reproduction conditions of each of the plurality of audio signals include a reproduction volume of the audio signal. In the above configuration, the reproduction volume of each of the plurality of audio signals is controlled. Therefore, a user of the terminal device can preferentially listen to sounds at one or more positions among the plurality of positions.

[0116] In a specific example (aspect 6) of aspect 4 or aspect 5, the playback control process includes an acoustic process that generates a playback signal by mixing the multiple acoustic signals according to the playback conditions, and causes the playback device to reproduce the sound represented by the playback signal.

[0117] In a specific example (Aspect 7) of any one of Aspects 1 to 5, the playback device includes a plurality of sound emitting units, and the playback conditions for each of the plurality of sound signals include designation of a sound emitting unit among the plurality of sound emitting units to which the sound signal is supplied. In the above configuration, the sound emitting unit to which each of the plurality of sound signals is supplied is controlled. Therefore, the user of the terminal device can clearly distinguish and listen to sounds at each of the plurality of positions.

[0118] In a specific example (Aspect 8) of any one of Aspects 1 to 7, the playback control process includes a synchronization process for aligning positions on a time axis between a first acoustic signal and a second acoustic signal among the plurality of acoustic signals. In the above configuration, the positions on a time axis between the first acoustic signal and the second acoustic signal among the plurality of acoustic signals are aligned. Therefore, a user can hear a natural sound in which the time difference between the first acoustic signal and the second acoustic signal is reduced.

[0119] In a specific example (aspect 9) of any one of aspects 1 to 8, the playback control information is generated in response to an instruction from a user of the terminal device. In the above configuration, the playback control information is generated in response to an instruction from a user of the terminal device. Therefore, playback in line with the preferences of the user is realized.

[0120] In a specific example (aspect 10) of any one of aspects 1 to 9, when acquiring the playback control information, one of a plurality of pieces of playback control information having different playback conditions for each of the audio signals is selected. In the above configuration, one of a plurality of pieces of playback control information prepared in advance is selected. Therefore, it is possible to diversify the sounds reproduced by the playback device.

[0121] In a specific example (Aspect 11) of Aspect 10, in acquiring the playback control information, in a first period, first playback control information of the plurality of pieces of playback control information is acquired, and in a second period different from the first period, second playback control information of the plurality of pieces of playback control information different from the first playback control information is acquired. In the above configuration, the first playback control information is selected in the first period, and the second playback control information is selected in the second period. Therefore, it is possible to realize diverse playback in which the playback conditions of the sounds represented by each acoustic signal change over time.

[0122] A control system according to one aspect (aspect 12) of the present disclosure includes a signal acquisition unit that acquires a plurality of sound signals generated for an event by collecting sound at different positions in parallel with the progress of the event, an information acquisition unit that acquires, for each of a plurality of terminal devices, playback control information that specifies playback conditions for each of the plurality of sound signals, and a playback control unit that executes a playback control process for each of the plurality of terminal devices, and in the playback control process, causes a playback device to play sounds corresponding to one or more of the plurality of sound signals in parallel with the progress of the event under playback conditions specified by the playback control information of the terminal device. A program according to another aspect of the present disclosure causes a computer to function as the control system according to aspect 12.

[0123] A terminal device according to one aspect (aspect 13) of the present disclosure includes a signal acquisition unit that acquires a plurality of sound signals generated for an event by collecting sound at different positions in parallel with the progress of the event, an information acquisition unit that acquires playback control information that specifies playback conditions for each of the plurality of sound signals, and a playback control unit that executes a playback control process, in which the playback control process causes a playback device to play back sounds corresponding to one or more of the plurality of sound signals in parallel with the progress of the event under the playback conditions specified by the playback control information. A program according to another aspect of the present disclosure causes a computer to function as the terminal device according to aspect 13. [Explanation of symbols]

[0124] 100...information system, 200...facility, 300...communication network, 11...recording system, 12...distribution system, 13...playback system, 14...control system, 15_1 to 15_N...sound collection unit, 21, 45...playback device, 23, 46...display device, 24, 47...sound emission device, 22...terminal device, 41, 51...control device, 42, 52...storage device, 43, 53...communication device, 44...operation device, 48...sound emission unit, 61, 71...information generation unit, 62, 72...signal acquisition unit, 63, 73...information acquisition unit, 64, 74...playback control unit, 65...situation analysis unit.

Claims

1. acquiring a plurality of acoustic signals generated by picking up sound at different locations for an event in parallel with the progress of the event; By analyzing information posted on a social networking site related to the progress of the event, a progress status of the event is estimated; acquiring, for each of a plurality of terminal devices, playback control information corresponding to the estimation result of the progress status, from among a plurality of pieces of playback control information specifying different playback conditions for each of the plurality of acoustic signals; execute a playback control process for each of the plurality of terminal devices; In the playback control process, sounds corresponding to one or more of the plurality of audio signals are played back on a playback device in parallel with the progress of the event under playback conditions designated by playback control information of the terminal device. A playback control method implemented by a computer system.

2. The different locations are locations present at the location where the event takes place. The playback control method according to claim 1.

3. The reproduction condition for each of the plurality of audio signals includes whether or not to reproduce the sound represented by the audio signal. The playback control method according to claim 1 or 2.

4. The reproduction conditions for each of the plurality of audio signals include a reproduction volume of the audio signal. The playback control method according to any one of claims 1 to 3.

5. The playback control process includes an audio process for generating a playback signal by mixing the multiple audio signals according to the playback conditions, and causes the playback device to play back the sound represented by the playback signal. The playback control method according to claim 3 or 4.

6. The playback device includes a plurality of sound emitting units, The reproduction condition for each of the plurality of sound signals includes designation of a sound emitting unit to which the sound signal is to be supplied among the plurality of sound emitting units. The playback control method according to any one of claims 1 to 5.

7. The reproduction control process includes a synchronization process for aligning positions on a time axis between a first acoustic signal and a second acoustic signal among the plurality of acoustic signals. The playback control method according to any one of claims 1 to 6.

8. a signal acquisition unit that acquires a plurality of acoustic signals generated for an event by collecting sounds at different positions in parallel with the progress of the event; a situation analysis unit that estimates a progress status of the event by analyzing posted information on a social networking site related to the progress status of the event; an information acquisition unit that acquires, for each of a plurality of terminal devices, playback control information corresponding to a result of the estimation of the progress state by the situation analysis unit, from a plurality of pieces of playback control information that specify different playback conditions for each of the plurality of acoustic signals; a playback control unit that executes a playback control process for each of the plurality of terminal devices; In the playback control process, sounds corresponding to one or more of the plurality of audio signals are played back on a playback device in parallel with the progress of the event under playback conditions designated by playback control information of the terminal device. Control system.

9. a signal acquisition unit that acquires a plurality of acoustic signals generated for an event by collecting sounds at different positions in parallel with the progress of the event; a situation analysis unit that estimates a progress status of the event by analyzing posted information on a social networking site related to the progress status of the event; an information acquisition unit that acquires, for each of a plurality of terminal devices, playback control information corresponding to a result of the estimation of the progress state by the situation analysis unit, from among a plurality of pieces of playback control information that specify different playback conditions for each of the plurality of acoustic signals; and a playback control unit that executes a playback control process for each of the plurality of terminal devices; A program for causing a computer to function as In the playback control process, sounds corresponding to one or more of the plurality of audio signals are played back on a playback device in parallel with the progress of the event under playback conditions designated by playback control information of the terminal device. program.

Images