Information processing device, information processing method, and program

Abstract

The present technology relates to an information processing device, an information processing method, and a program that make it possible to suitably provide the audio of a content to a plurality of users. An information processing device according to the present technology comprises a reproduction control unit that, on the basis of a measurement result of a first delay from when a first reproduction device starts reproduction processing of a content until the audio of the content is output from a first output device, controls a start time at which the first reproduction device starts the reproduction processing of the content. The present technology can be applied, for example, to a content reproduction system in which a plurality of smartphones simultaneously provide the same audio.

Description

Information processing device, information processing method, and program

[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly to an information processing device, an information processing method, and a program that are capable of preferably providing audio of content to multiple users.

[0002] When multiple video contents are simultaneously screened in the same room, there is a technique for exclusively providing the audio of the video content that each user wants to view so that others cannot hear it (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-167669

[0004] In the technology described in Patent Document 1, playback processing of multiple video contents is performed by a single video and audio management device, whereas consider a case where playback processing of the same content is performed by, for example, a smartphone owned by multiple users, and the audio of the content is provided to each user.

[0005] In this case, even if the content playback process starts simultaneously, the communication delay and playback delay that occur when providing the content audio differ from smartphone to smartphone, resulting in a difference in the timing at which the content audio is actually provided between smartphones. This impairs the quality of the shared experience between multiple users, such as users chatting while enjoying the content or sharing their impressions of the content.

[0006] The present technology has been made in view of such circumstances, and makes it possible to provide audio of content to a plurality of users in a suitable manner.

[0007] An information processing device according to one aspect of the present technology includes a playback control unit that controls the start time at which a first playback device starts playback processing of the content, based on a measurement result of a first delay that occurs between when the first playback device starts playback processing of the content and when audio of the content is output from a first output device.

[0008] An information processing method according to one aspect of the present technology includes controlling the start time at which a playback device starts playback processing of a content based on a measurement result of a delay between when the playback device starts playback processing of the content and when the audio of the content is output from an output device.

[0009] A program according to one aspect of the present technology causes a computer to execute a process that includes controlling the start time at which the playback device starts the playback process of the content, based on a measurement result of the delay between when the playback device starts the playback process of the content and when the audio of the content is output from an output device.

[0010] In one aspect of the present technology, the start time at which the playback device starts the playback process of the content is controlled based on the measurement results of the delay between when the playback device starts the playback process of the content and when the audio of the content is output from the output device.

[0011] 1 is a diagram illustrating an example configuration of a content playback system according to an embodiment of the present technology. FIG. 1 is a diagram illustrating a delay that occurs when providing audio of content. FIG. 2 is a diagram illustrating a flow of compensation for communication delay. FIG. 3 is a first diagram illustrating a flow of compensation for playback delay. FIG. 4 is a second diagram illustrating a flow of compensation for playback delay. FIG. 5 is a diagram illustrating an example of a relative delay based on a relative delay of a leader terminal and a waiting time for playback delay compensation. FIG. 6 is a first diagram illustrating an example of a screen transition displayed on a smartphone when calculating a waiting time for playback delay compensation. FIG. 7 is a second diagram illustrating an example of a screen transition displayed on a smartphone when calculating a waiting time for playback delay compensation. FIG. 8 is a diagram illustrating an example of a screen transition displayed on a smartphone when outputting audio of content. FIG. 9 is a block diagram illustrating an example configuration of a leader terminal. FIG. 10 is a block diagram illustrating an example configuration of a follower terminal. FIG. 11 is a sequence diagram illustrating processing performed by a content playback system. FIG. 12 is a diagram illustrating a first application example of the content playback system of the present technology. FIG. 13 is a diagram illustrating a second application example of the content playback system of the present technology. FIG. 14 is a diagram illustrating a third application example of the content playback system of the present technology. FIG. 15 is a block diagram illustrating a detailed configuration example of a content playback system when linking a moving image displayed on a display device with audio output from an output device of a smartphone.

[0012] Hereinafter, embodiments of the present technology will be described in the following order: 1. Configuration of a content playback system 2. Configuration and operation of each device 3. Application examples

[0013] 1. Configuration of Content Reproduction System FIG. 1 is a diagram showing an example of the configuration of a content reproduction system according to an embodiment of the present technology.

[0014] The content playback system shown in Fig. 1 is a system in which multiple users, each of whom owns a smartphone, play the same content and provide each user with the audio of the content. The audio here is not limited to the voice of a person or character, but may include any sound such as music, sound effects, or background sounds.

[0015] The content playback system of FIG. 1 is configured by an NTP server 1, a trigger server 2, and smartphones 3A to 3C, which are interconnected via a network 11 such as the Internet.

[0016] An NTP (Network Time Protocol) server 1 provides NTP time, which is time information in the UTC (Coordinated Universal Time) time format, via a network 11. A trigger server 2 and smartphones 3A to 3C operate in synchronization with the NTP time provided by the NTP server 1.

[0017] The trigger server 2 transmits a playback trigger to the smartphones 3A to 3C to instruct them to start the playback process of the content.

[0018] The NTP server 1 and the trigger server 2 may be configured as a single server.

[0019] The smartphones 3A to 3C are information processing devices that receive a playback trigger transmitted from the trigger server 2 and perform playback processing of the same content in accordance with the playback trigger.

[0020] In the example of Fig. 1, smartphone 3A is a smartphone owned by user A, and outputs the audio of the content from a speaker 21A (not shown) mounted on smartphone 3A. Smartphone 3B is a smartphone owned by user B, and outputs the audio of the content from earphones (in-ear headphones) 21B connected to smartphone 3B by wire. Smartphone 3C is a smartphone owned by user C, and outputs the audio of the content from headphones 21C connected wirelessly to smartphone 3C.

[0021] For example, users A to C are users who experience the same augmented reality (AR) space nearby, and view content placed in the AR space via their own smartphones. The AR space is a space that combines real space and virtual space. Users A to C can have a shared experience with multiple people, such as chatting with each other while enjoying the content and sharing their impressions of the content.

[0022] Hereinafter, when there is no need to particularly distinguish between the smartphones 3A to 3C, they will be simply referred to as smartphones 3. Furthermore, when there is no need to particularly distinguish between the speaker 21A, the earphone 21B, and the headphones 21C, they will be simply referred to as output devices 21. Note that the information processing device that performs the playback process of content may be configured as a tablet terminal, a wearable device, a PC, a game console, or the like.

[0023] When the NTP server 1 and the trigger server 2 are placed on the same network as the smartphone 3, the NTP server 1 and the trigger server 2 may be connected to the smartphone 3 via a router or the like, without going through the network 11. A system in which the NTP server 1 and the trigger server 2 are connected to the smartphone 3 without going through the network 11 is suitable for use cases in specific locations such as concert venues and art museums, or for use cases requiring high confidentiality such as business use.

[0024] FIG. 2 is a diagram illustrating a delay that occurs when providing audio of content.

[0025] In the content playback system of the present technology, the provision of the audio of the content is realized by the smartphone 3 receiving a playback trigger transmitted from the trigger server 2 via a communication path, as shown in the upper part of Figure 2, and by the smartphone 3 performing playback processing of the content in accordance with the playback trigger and outputting the audio of the content from the output device 21.

[0026] Here, a communication delay occurs between when the playback trigger is transmitted and when it is received, and this delay varies depending on the communication path, etc. Also, a playback delay occurs between when the smartphone 3 starts playback processing of the content and when the audio of the content is output from the output device 21, and this delay varies depending on the performance of the smartphone 3, the connection method of the output device 21, etc.

[0027] Since the communication path, the performance of the smartphone 3, the connection method of the output device 21, etc. differ for each smartphone 3 (each user), the communication delay and playback delay that occur when providing the audio of the content differ for each smartphone 3.

[0028] For example, as shown in the lower part of Figure 2, we compare the time (delay) required to provide the audio of content between a high-end model smartphone 3 that communicates with the trigger server 2 using a high-speed line and is connected to earphones via a wired connection, and a low-end model smartphone 3 that communicates with the trigger server 2 using a low-speed line and is connected to earphones wirelessly.

[0029] A smartphone 3 that communicates with the trigger server 2 using a high-speed line has a smaller communication delay than a smartphone 3 that communicates with the trigger server 2 using a low-speed line. Also, a high-end model smartphone 3 that is connected to earphones by wire has a smaller playback delay than a low-end model smartphone 3 that is connected to earphones wirelessly.

[0030] Therefore, there is a difference in the time it takes for the audio of the content to be output between smartphones, as indicated by the double-headed arrows at the bottom of Figure 2. This causes a difference in the timing at which the audio of the content is provided between smartphones, impairing the quality of the shared experience among multiple people.

[0031] Therefore, the content playback system of the present technology synchronizes the timing at which the audio of the content is provided to each user so that multiple users feel as if they are listening to the same audio at the same time by compensating for communication delays and playback delays for each smartphone 3. The audio of the content is provided to each user with a time difference of, for example, 100 ms or less so that multiple users feel as if they are listening to the same audio at the same time.

[0032] FIG. 3 is a diagram illustrating the flow of compensation for communication delay.

[0033] As described above, the communication delay is the time required to transmit a playback trigger between the trigger server 2 and the smartphone 3. In the content playback system of the present technology, the round-trip time (RTT) that is the round-trip time for communication between the smartphone 3 and the trigger server 2 is measured for each smartphone 3, and the communication delay is compensated for based on the RTT for each smartphone 3.

[0034] To compensate for communication delays, first, as shown in the upper part of Fig. 3, smartphones 3A to 3C synchronize their system times with the NTP time provided by the NTP server 1. Because all smartphones 3 operate in synchronization with the NTP time, the time discrepancy between the smartphones 3 is minimized.

[0035] Next, as shown at the tip of the white arrow #11 in Fig. 3, for example, smartphones 3A to 3C measure the RTT of their own devices. In the example of Fig. 3, the RTT of smartphone 3A is 120 ms, the RTT of smartphone 3B is 10 ms, and the RTT of smartphone 3C is 70 ms. The time obtained by halving the RTT corresponds to the communication delay of each smartphone 3. Smartphones 3A to 3C notify the trigger server 2 of the measured RTT.

[0036] Next, the trigger server 2 calculates, for each smartphone 3, a waiting time from when the playback trigger is received until when audio output starts, based on the RTT measured for each smartphone 3. This waiting time can be considered a waiting time for compensating for communication delays. The trigger server 2 notifies each smartphone 3 of the waiting time for compensating for communication delays. In the example of FIG. 3 , the waiting time for smartphone 3A is set to 0 ms, the waiting time for smartphone 3B is set to 55 ms, and the waiting time for smartphone 3C is set to 25 ms.

[0037] Next, as shown at the tip of the hollow arrow #12 in Fig. 3 , the trigger server 2 sets a time after the current time (future) by the largest communication delay among the communication delays of each smartphone 3 as the time to start outputting the content (output start time), and transmits a playback trigger indicating the output start time to each smartphone 3. In the example of Fig. 3 , since the communication delay of smartphone 3A is 60 ms, a time 60 ms after the current time is set as the output start time.

[0038] Each smartphone 3 starts outputting the audio of the content after waiting for a waiting time after receiving the playback trigger transmitted from the trigger server 2. In the example of Fig. 3, smartphone 3A starts outputting the audio of the content immediately after receiving the playback trigger. Smartphone 3B starts outputting the audio of the content 55 ms after receiving the playback trigger, and smartphone 3C starts outputting the audio of the content 25 ms after receiving the playback trigger.

[0039] In this way, the audio delay between the smartphones 3 caused by the difference in communication delay is compensated for.

[0040] 4 and 5 are diagrams for explaining the flow of compensation for playback delay.

[0041] As described above, the playback delay is the time from when the playback process of content starts until audio is actually output, and varies depending on the performance of the smartphone 3 and the connection method of the output device. In particular, it is known that there is a large difference in playback delay between earphones connected via a wired connection and earphones connected via Bluetooth (registered trademark).

[0042] In the content playback system of the present technology, a leader terminal and follower terminals are set, and a relative playback delay (relative delay) based on the playback delay of the leader terminal is measured for each follower terminal. For example, smartphone 3A is the leader terminal, and smartphone 3B is the follower terminal. Hereinafter, smartphone 3A is also referred to as leader terminal 3A, and smartphone 3B is also referred to as follower terminal 3B. Based on the relative playback delay measured for each follower terminal, compensation for the playback delay of each smartphone 3 is performed.

[0043] To compensate for the playback delay, first, as shown in the upper part of Fig. 4, the leader terminal 3A and the follower terminal 3B synchronize their respective system times with the NTP time provided by the NTP server 1. Here, the follower terminal 3B is located near the leader terminal 3A.

[0044] Next, as indicated by the tip of the white arrow #21 in FIG. 4, the reproduction delay of the reader terminal 3A is measured.

[0045] Specifically, the trigger server 2 notifies the leader terminal 3A and the follower terminal 3B of a playback start time T1, which is the time when the leader terminal 3A starts the playback process of the test sound. To allow for communication delays, the time it takes for the follower terminal 3B to start recording, and the time it takes for the leader terminal 3A to output the test sound, the playback start time T1 is set to, for example, a time several hundred milliseconds to several seconds after the current time.

[0046] The leader terminal 3A starts the playback process of the test sound at playback start time T1 and outputs the test sound from a speaker mounted on the leader terminal 3A. The follower terminal 3B picks up the test sound with a microphone mounted on the follower terminal 3B and records it. During recording, the follower terminal 3B embeds a pattern (flag information) indicating the playback start time T1 in the recorded data.

[0047] 4, the follower terminal 3B analyzes the waveform of the recorded data to calculate a difference time D1 from the playback start time T1 to the time the test sound is picked up by the microphone. The difference time D1 corresponds to the playback delay of the leader terminal 3A.

[0048] Next, as shown in the upper part of FIG. 5, the reproduction delay of the follower terminal 3B is measured.

[0049] Specifically, the follower terminal 3B starts the playback process of the test sound at an arbitrary playback start time T2 and outputs the test sound from the earphone 21B connected to the follower terminal 3B. The follower terminal 3B picks up the test sound with a microphone mounted on the follower terminal 3B and records it. During recording, the follower terminal 3B embeds a pattern (flag information) indicating the playback start time T2 in the recorded data.

[0050] 5, the follower terminal 3B calculates a differential time D2 from the playback start time T2 to the time the test sound is picked up by the microphone by analyzing the waveform of the recorded data. The differential time D2 corresponds to the playback delay of the follower terminal 3B. The follower terminal 3B notifies the trigger server 2 of the differential time D1 and the differential time D2.

[0051] Next, the trigger server 2 calculates the difference between the differential time D1 (playback delay of the leader terminal 3A) and the differential time D2 (playback delay of the follower terminal 3B) as the relative delay of the follower terminal 3B based on the playback delay of the leader terminal 3A. Based on the relative delay of the follower terminal 3B, the trigger server 2 calculates, for each smartphone 3, the waiting time from the playback reference time that serves as the reference for content playback until the content playback process actually starts. This waiting time can be said to be a waiting time for playback delay compensation. The trigger server 2 notifies each smartphone 3 of the waiting time for playback delay compensation. In the example of FIG. 5 , the waiting time of the leader terminal 3A is set to Ws, and the waiting time of the follower terminal 3B is set to 0 s.

[0052] Next, as indicated by the tip of the hollow arrow #22 in Figure 5, the trigger server 2 transmits a playback trigger indicating the playback reference time to each smartphone 3. Each smartphone 3 receives the playback trigger transmitted from the trigger server 2, waits for the waiting time from the playback reference time indicated by the playback trigger, and then starts playback processing of the content. In the example of Figure 3, the leader terminal 3A starts playback processing of the content at a time Ws after the playback reference time and outputs the audio of the content from the speaker. The follower terminal 3B starts playback processing of the content at the playback reference time and outputs the audio of the content from the earphone 21B.

[0053] For example, consider a case where compensation for playback delay is performed within a group consisting of a leader terminal and four smartphones, follower terminals A to C, as shown in Fig. 6. As shown in Fig. 6A, the relative delay of the leader terminal is 0 ms, the relative delay of follower terminal A is 100 ms, the relative delay of follower terminal B is 200 ms, and the relative delay of follower terminal C is -100 ms.

[0054] In this case, when each smartphone 3 starts the content playback process simultaneously at the playback reference time without compensating for the playback delay, audio is output from the output device 21 in the order of follower terminal C, leader terminal, follower terminal A, and follower terminal B.

[0055] If the relative delay of each smartphone 3 based on the playback delay of the leader terminal is D and the maximum value of the relative delay within the group is Dmax, then the waiting time of each smartphone 3 is W = Dmax - D, and the timing at which audio is actually provided to each user is synchronized.

[0056] Specifically, as shown in B of Figure 6, since the relative delay D of the leader terminal is 0 ms and Dmax is 200 ms, the leader terminal starts content playback processing at 200-0 = 200 [ms] after the playback reference time. Similarly, follower terminal A starts content playback processing at 200-100 = 100 [ms] after the playback reference time, and follower terminal B starts content playback processing at 200-200 = 0 [ms] after the playback reference time. Follower terminal C starts content playback processing at 200-(-100) = 300 [ms] after the playback reference time.

[0057] As described above, by adding each waiting time W to the playback reference time, audio discrepancies caused by differences in playback delays between the smartphones 3 are compensated for. By compensating for playback delays, all users can share the same audio experience, even if they are using different types of smartphones 3 and output devices 21.

[0058] If the minimum value of the relative delay within the group is Dmin, then by having each smartphone 3 start content playback processing at time (T-D+Dmin), it is also possible to simultaneously output audio from the output device at a time close to the playback reference time T. Furthermore, if the absolute value (measured value) of the playback delay of each smartphone 3 is Dabs, then it is also possible to simultaneously output audio from the output device at the playback reference time T by having each smartphone 3 start content playback processing at time (T-Dabs).

[0059] 7 and 8 are diagrams showing examples of screen transitions displayed on the smartphone 3 when calculating the waiting time W for compensating for playback delay.

[0060] When a user launches, for example, a content playback application on each smartphone 3, a screen shown on the left side of FIG. 7 is displayed on each smartphone 3 (leader terminal and follower terminal).

[0061] To calculate the waiting time W, first, the group (group A) to which the leader terminal and follower terminal belong is displayed in the upper part of the screen of the leader terminal and the follower terminal, as shown on the left side of Fig. 7. In the lower part of the screen of the leader terminal and the follower terminal, a button B1 for selecting to connect the smartphone 3 to the trigger server 2 as the leader terminal, and a button B2 for selecting to connect the smartphone 3 to the trigger server 2 as the follower terminal are displayed.

[0062] The user of the leader terminal taps button B1, and the user of the follower terminal taps button B2.

[0063] When button B2 is tapped, the follower terminal connects as a follower terminal to the trigger server 2. After the follower terminal is connected to the trigger server 2, an additional message is displayed in the upper part of the screen of the follower terminal indicating that the follower terminal has already connected to the trigger server 2 as a follower terminal, and in the lower part, buttons B1 and B2 disappear and button B3 for starting measurement of the playback delay of the leader terminal is displayed.

[0064] When a user of the follower terminal taps button B3, the follower terminal waits until the leader terminal connects to the trigger server 2. While the follower terminal is waiting, button B3 disappears and a label L1 containing the text "Waiting for leader" is displayed in the lower part of the screen of the follower terminal.

[0065] On the other hand, when button B1 is tapped, the leader terminal connects as the leader terminal to the trigger server 2. After the leader terminal is connected to the trigger server 2, an additional message is displayed in the upper part of the screen of the follower terminal indicating that the follower terminal is already connected to the trigger server 2 as the leader terminal, and in the lower part, buttons B1 and B2 disappear and button B3 is displayed.

[0066] When the user of the leader terminal taps button B3, the trigger server transmits test sound playback start time T1 to the leader terminal and follower terminals, and the follower terminals start recording. When recording starts, label L1 disappears and label L2 containing the text "Measuring" is displayed in the lower part of the follower terminal's screen, as shown in the lower left of Figure 8. The leader terminal starts the test sound playback process at playback start time T1. When the test sound playback process starts, button B3 disappears and label L3 containing the text "Playing" is displayed in the lower part of the leader terminal's screen, as shown in the upper left of Figure 8.

[0067] To measure the playback delay of the leader terminal, the user of the leader terminal taps button B3 after all follower terminals are in a standby state so that the test sound output from the speaker of the leader terminal can be reliably picked up by the microphones of all follower terminals.

[0068] After the follower terminal has completed measuring the playback delay (difference time D1) of the leader terminal, the upper part of the follower terminal's screen additionally displays a message indicating that measurement of the playback delay of the leader terminal has been completed, and in the lower part, label L2 disappears and button B4 for starting measurement of the playback delay of the follower terminal is displayed. Also, in the upper part of the leader terminal's screen, the group to which the leader terminal belongs (group A) and a message indicating that the device is in a standby state for a playback trigger are displayed, and in the lower part, label L3 disappears and button B5 for ending the content playback application is displayed.

[0069] When the user of the follower terminal taps button B4, the follower terminal starts recording and begins playback of the test sound at playback start time T2. Once recording starts, button B4 disappears from the lower part of the follower terminal's screen, and label L2 is displayed again. After the follower terminal has finished measuring its own playback delay (difference time D2), the upper part of the follower terminal's screen displays the group to which the follower terminal belongs (group A) and a message that the follower terminal is waiting for a playback trigger, and label L2 disappears from the lower part, and button B5 is displayed.

[0070] FIG. 9 is a diagram showing an example of screen transitions displayed on the smartphone 3 when audio of content is output.

[0071] When outputting the audio of the content, the leader terminal and the follower terminal are not distinguished. In the example of Fig. 9, an example will be described in which the audio of the content is simultaneously output from a low-delay terminal that does not cause much delay in outputting the audio of the content and a high-delay terminal that causes a large delay.

[0072] As explained with reference to the right side of Figure 8, after the low-latency terminal and the high-latency terminal enter a standby state for a playback trigger, at time T11, the trigger server 2 sends a playback trigger indicating the playback reference time to the low-latency terminal and the high-latency terminal.

[0073] At time T12, the low-latency terminal receives the playback trigger transmitted from the trigger server 2. When the playback trigger is received, the message that the low-latency terminal is in a standby state for the playback trigger disappears from the upper part of the screen of the low-latency terminal, and a message that the playback trigger has been received and that the low-latency terminal is in a standby state for playback processing of the content is displayed instead.

[0074] At time T13 after time T12, the high delay terminal receives the playback trigger transmitted from the trigger server 2. When the playback trigger is received, the message that the high delay terminal is in a standby state for the playback trigger disappears from the upper part of the screen of the high delay terminal, and a message that the playback trigger has been received and that the high delay terminal is in a standby state for playback processing of the content is displayed instead.

[0075] The high latency terminal starts the content playback process at time T14, which is the playback reference time, for example. When the content playback process starts, the message in the upper part of the screen of the high latency terminal disappears indicating that the device is in a standby state for content playback process, and a message indicating that playback process has started is displayed instead.

[0076] The low-latency terminal starts the content playback process at time T15 after time T14. When the content playback process starts, the message in the upper part of the screen of the low-latency terminal disappears indicating that the device is in a standby state for content playback process, and a message indicating that playback process has started is displayed instead.

[0077] At time T16, the audio of the content is simultaneously output from, for example, a speaker mounted on the low-delay terminal and the output device 21 connected to the long-delay terminal.

[0078] In this way, in a low-latency terminal (a smartphone 3 with a small relative delay D), content playback processing is started at a timing close to the timing when the audio is actually output, and in a high-latency terminal (a smartphone 3 with a large relative delay D), content playback processing is started at a timing much earlier than the timing when the audio is actually output. This makes it possible to simultaneously provide the audio of the content to users of low-latency terminals and high-latency terminals.

[0079] 2. Configuration and Operation of Each Device FIG. 10 is a block diagram showing an example of the configuration of the reader terminal 3A.

[0080] As shown in FIG. 10, the reader terminal 3A includes a communication unit 71, a control unit 72, a storage unit 73, and an audio playback unit 74.

[0081] The communication unit 71 is a wireless communication module that performs wireless communication with external devices such as the NTP server 1 and the trigger server 2. The communication unit 71 communicates with the external devices, acquires content data and NTP time, and supplies them to the control unit 72. The communication unit 71 notifies the control unit 72 of the playback start time T1 of the test sound and the waiting time for playback delay compensation that have been notified by the trigger server 2. The communication unit 71 receives a playback trigger transmitted from the trigger server 2, and supplies them to the control unit 72.

[0082] The control unit 72 is configured with a CPU and the like, and controls the entire reader terminal 3 A. For example, the control unit 72 synchronizes the system time of the reader terminal 3 A with the NTP time supplied from the communication unit 71. The control unit 72 also stores the content data supplied from the communication unit 71 in the storage unit 73.

[0083] The control unit 72 has a reproduction control unit 81. The reproduction control unit 81 controls the start time at which the audio reproduction unit 74 starts the reproduction process of the content, based on the measurement result of the reproduction delay of the reader terminal 3A.

[0084] Specifically, the playback control unit 81 acquires audio data of the test sound from the storage unit 73 and supplies the audio data of the test sound to the audio playback unit 74. The playback control unit 81 causes the audio playback unit 74 to start playback processing of the test sound at the playback start time T1 notified by the communication unit 71.

[0085] Furthermore, the playback control unit 81 acquires content data from the storage unit 73 and supplies audio data included in the content to the audio playback unit 74. For example, the playback control unit 81 waits for a waiting time from the playback reference time indicated by the playback trigger supplied from the communication unit 71, and then causes the audio playback unit 74 to start playback processing of the content.

[0086] Note that some of the functions of the playback control unit 81 may be realized by the trigger server 2. In this case, the playback control unit of the trigger server 2 sets the playback start time of the content for each smartphone 3, taking into account a waiting time for compensating for a playback delay, and transmits a playback trigger indicating the playback start time of the content to each smartphone 3.

[0087] In the reader terminal 3 A, the sound of the content is output from one of the sound reproducing unit 74 , the earphone 51 , and the BT device 52 .

[0088] The audio reproduction unit 74 includes an audio buffer 91 , a DA (Digital to Analog) converter 92 , an audio output unit 93 , and a BT transmission unit 94 .

[0089] The audio buffer 91 buffers the audio data supplied from the playback control unit 81. The audio buffer 91 supplies the audio data of the test sound to the DA converter 92, for example, at playback start time T1, in accordance with the control of the playback control unit 81. The audio buffer 91 waits, for example, for a waiting time from the playback reference time, in accordance with the control of the playback control unit 81, and then supplies the audio data of the content to the DA converter 92.

[0090] The DA converter 92 converts the audio data supplied from the audio buffer 91 into an analog audio signal, and supplies the audio signal to an audio output unit 93 or the earphone 51 .

[0091] The audio output unit 93 is configured as, for example, a speaker mounted on the reader terminal 3 A. The audio output unit 93 outputs the audio represented by the audio signal supplied from the DA converter 92 .

[0092] The earphone 51 is an output device connected by wire to the reader terminal 3 A. The earphone 51 outputs the sound represented by the audio signal supplied from the DA converter 92.

[0093] The BT transmission unit 94 is a wireless communication module that uses BT to perform wireless communication with the BT device 52. The BT transmission unit 94 transmits the audio data supplied from the audio buffer 91 to the BT device 52.

[0094] The BT device 52 is an output device connected to the reader terminal 3 A via BT. The BT device 52 is made up of a BT receiving unit 111, an audio buffer 112, a DA converter 113, and an audio output unit 114.

[0095] The BT receiving unit 111 is a wireless communication module that uses BT to perform wireless communication with the leader terminal 3 A. The BT receiving unit 111 receives voice data transmitted from the leader terminal 3 A and supplies the voice data to the voice buffer 112 .

[0096] The audio buffer 112 buffers the audio data supplied from the BT receiving unit 111 and supplies the audio data to a DA converter 113 .

[0097] The DA converter 113 converts the audio data supplied from the audio buffer 112 into an analog audio signal, and supplies the audio signal to an audio output unit 114 .

[0098] The audio output unit 114 outputs the audio represented by the audio signal supplied from the DA converter 113 .

[0099] FIG. 11 is a block diagram showing an example of the configuration of the follower terminal 3B.

[0100] As shown in FIG. 11, the follower terminal 3B includes a communication unit 171, a control unit 172, a storage unit 173, a recording unit 174, and an audio playback unit 175.

[0101] The communication unit 171 is a wireless communication module that performs wireless communication with external devices such as the NTP server 1 and the trigger server 2. The communication unit 171 communicates with the external devices, acquires content data and NTP time, and supplies them to the control unit 172. The communication unit 171 notifies the control unit 172 of the playback start time T1 of the test sound and the waiting time for playback delay compensation, which have been notified by the trigger server 2. The communication unit 171 receives a playback trigger transmitted from the trigger server 2, and supplies them to the control unit 172.

[0102] The communication unit 171 notifies the trigger server 2 of a differential time D1 corresponding to the reproduction delay of the leader terminal 3A and a differential time D2 corresponding to the reproduction delay of the follower terminal 3B, both measured by the control unit 172.

[0103] The control unit 172 is configured with a CPU and the like, and controls the entire follower terminal 3B. For example, the control unit 172 synchronizes the system time of the follower terminal 3B with the NTP time supplied from the communication unit 171. The control unit 172 stores the content data supplied from the communication unit 171 in the storage unit 173. The control unit 172 causes the recording unit 174 to start recording at a time before the playback start time T1 notified by the communication unit 171.

[0104] The control unit 172 has a calculation unit 181 and a playback control unit 182. The calculation unit 181 calculates differential time D1 by analyzing the waveform of the audio recording data obtained by the recording unit 174 recording the test sound output from the output device of the leader terminal 3A. The calculation unit 181 also calculates differential time D2 by analyzing the waveform of the audio recording data obtained by the recording unit 174 recording the test sound output from the output device of the follower terminal 3B.

[0105] The calculation unit 181 notifies the communication unit 171 of the difference time D1 and the difference time D2.

[0106] The playback control unit 182 controls the start time at which the audio playback unit 175 starts the playback process of the content, based on the measurement result of the playback delay of the follower terminal 3B.

[0107] Specifically, the playback control unit 182 acquires test sound data from the storage unit 173 and supplies the audio data of the test sound to the audio playback unit 175. The playback control unit 182 causes the audio playback unit 175 to start playback processing of the test sound at a predetermined playback start time T2.

[0108] Furthermore, the playback control unit 182 acquires content data from the storage unit 173 and supplies audio data included in the content to the audio playback unit 175. For example, the playback control unit 182 waits for a waiting time from the playback reference time indicated by the playback trigger supplied from the communication unit 171, and then causes the audio playback unit 175 to start playback processing of the content.

[0109] Note that part of the functions of the playback control unit 182 may be realized by the trigger server 2. In this case, the playback control unit of the trigger server 2 sets the playback start time of the content for each smartphone 3, taking into account a waiting time for compensating for a playback delay, and transmits a playback trigger indicating the playback start time of the content to each smartphone 3.

[0110] In the follower terminal 3B, the sound of the content is output from one of the sound reproducing unit 175, the earphone 151, and the BT device 152.

[0111] The recording unit 174 includes a microphone 191 , an AD (Analog to Digital) converter 192 , a recording buffer 193 , and a recording processing unit 194 .

[0112] The microphone 191 picks up and records the test sound output from the output device of the leader terminal 3 A or the follower terminal 3 B, and obtains an analog recording signal. The microphone 191 supplies the recording signal to an AD converter 192.

[0113] The AD converter 192 converts the analog recording signal supplied from the microphone 191 into recording data, and supplies it to a recording buffer 193 .

[0114] The recording buffer 193 buffers the recording data supplied from the AD converter 192 and supplies the recording data to a recording processing unit 194 .

[0115] When the recording processing unit 194 measures the playback delay of the reader terminal 3A, it embeds a pattern indicating the playback start time T1 into the recording data supplied from the recording buffer 193. When the recording processing unit 194 measures the playback delay of the reader terminal 3A, it embeds a pattern indicating the playback start time T2 into the recording data supplied from the recording buffer 193.

[0116] The recording processing unit 194 supplies the recording data with the embedded pattern to the control unit 172 .

[0117] The audio reproduction unit 175 includes an audio buffer 201 , a DA converter 202 , an audio output unit 203 , and a BT transmission unit 204 .

[0118] The audio buffer 201 buffers the audio data supplied from the playback control unit 182. The audio buffer 201 supplies the audio data of the test sound to the DA converter 202, for example, at playback start time T2, in accordance with the control of the playback control unit 182. The audio buffer 201 waits, for example, for a waiting time from the playback reference time, in accordance with the control of the playback control unit 182, and then supplies the audio data to the DA converter 202.

[0119] The DA converter 202 converts the audio data supplied from the audio buffer 201 into an analog audio signal, and supplies the audio signal to an audio output unit 203 or the earphone 151 .

[0120] The audio output unit 203 is configured as, for example, a speaker mounted on the follower terminal 3 B. The audio output unit 203 outputs the audio represented by the audio signal supplied from the DA converter 202 .

[0121] The earphone 151 is an output device that is connected by wire to the follower terminal 3 B. The earphone 151 outputs the sound represented by the audio signal supplied from the DA converter 202 .

[0122] The BT transmission unit 204 is a wireless communication module that uses BT to perform wireless communication with the BT device 152. The BT transmission unit 204 transmits the audio data supplied from the audio buffer 201 to the BT device 152.

[0123] The BT device 152 is an output device connected to the follower terminal 3 B via BT. The BT device 152 is made up of a BT receiving unit 221 , an audio buffer 222 , a DA converter 223 , and an audio output unit 224 .

[0124] The BT receiving unit 221 is a wireless communication module that uses BT to perform wireless communication with the follower terminal 3 B. The BT receiving unit 221 receives voice data transmitted from the follower terminal 3 B and supplies the voice data to the voice buffer 222 .

[0125] The audio buffer 222 buffers the audio data supplied from the BT receiving unit 221 and supplies the audio data to a DA converter 223 .

[0126] The DA converter 223 converts the audio data supplied from the audio buffer 222 into an analog audio signal, and supplies the audio signal to an audio output unit 224 .

[0127] The audio output unit 224 outputs the audio represented by the audio signal supplied from the DA converter 223 .

[0128] Next, the processing performed by the content playback system having the above configuration will be described with reference to the sequence diagram of Fig. 12. In Fig. 12, the description will be given assuming that the NTP server 1 and the trigger server 2 are configured as a single server.

[0129] In step S1, the server provides the leader terminal 3A and the follower terminal 3B with the NTP time, and the control unit 72 of the leader terminal 3A and the control unit 172 of the follower terminal 3B synchronize their own system times with the NTP time.

[0130] In step S11, the server notifies the leader terminal 3A and the follower terminal 3B of the playback start time T1.

[0131] In step S51 (at a time before the playback start time T1), the recording unit 174 of the follower terminal 3B starts recording.

[0132] In step S31 (at playback start time T1), the audio playback unit 175 of the leader terminal 3A starts playback of the test sound. The recording unit 174 of the follower terminal 3B collects and records the test sound output from the output device of the leader terminal 3A.

[0133] In step S52, the calculation unit 181 of the follower terminal 3B analyzes the waveform of the recorded data and calculates the differential time D1. If multiple follower terminals 3B are provided in the content playback system, the recording of the test audio and the calculation of the differential time D1 may be performed simultaneously on all follower terminals 3B or individually on each follower terminal 3B.

[0134] In step S53 (at a time before the playback start time T2), the recording unit 174 of the follower terminal 3B starts recording.

[0135] In step S54 (at playback start time T2), the audio playback unit 175 of the follower terminal 3B starts playback processing of the test sound. The recording unit 174 of the follower terminal 3B picks up and records the test sound output from the output device of the follower terminal 3B.

[0136] In step S55, the calculation unit 181 of the follower terminal 3B analyzes the waveform of the recorded data and calculates the differential time D2. When multiple follower terminals 3B are provided in the content playback system, the output of the test audio, the recording of the test audio, and the calculation of the differential time D2 may be performed simultaneously in all of the follower terminals 3B, or may be performed individually.

[0137] In step S56, the communication unit 171 of the follower terminal 3B notifies the server of the differential time D1 and the differential time D2.

[0138] In step S12, the server calculates a waiting time for compensating for a reproduction delay. For example, it is assumed that the waiting time of the leader terminal 3A is 0 ms and the waiting time of the follower terminal 3B is W ms.

[0139] In step S13, the server notifies the follower terminal 3B of the waiting time W.

[0140] In step S14, the server transmits a playback trigger indicating the playback reference time T to the leader terminal 3A and the follower terminal 3B. The communication unit 71 of the leader terminal 3A and the follower terminal 3B 171 receive the playback trigger transmitted from the server.

[0141] In step S32 (reproduction reference time T), the audio reproduction unit 74 of the reader terminal 3A starts the reproduction process of the content.

[0142] In step S57 (at time (T+W)), the audio playback unit 175 of the follower terminal 3B starts the playback process of the content.

[0143] As described above, in the smartphone 3 (playback device), the start time at which the smartphone 3 starts the content playback process is controlled based on the measurement result of the playback delay required from when the smartphone 3 starts the content playback process until the audio of the content is output from the output device. In particular, in the follower terminal 3B (first playback device), the start time at which the content playback process is controlled is based on the measurement result of the playback delay (first delay) of the follower terminal 3B and the measurement result of the playback delay (second delay) required from when the leader terminal 3A (second playback device) starts the content playback process until the audio of the content is output from the output device (second output device).

[0144] The start time of the content playback process in the follower terminal 3B is controlled based on the playback delay in the leader terminal 3A and the playback delay in the follower terminal 3B, making it possible to provide the same audio to multiple users simultaneously.

[0145] 3. Application Examples Interlocking with a performance performed by equipment installed in a facility FIG. 13 is a diagram showing a first application example of a content playback system according to the present technology.

[0146] Audio linked to a performance (reference content) performed by equipment such as illumination, in-house broadcasting, and lighting installed in a facility such as a theme park may be provided by the smartphone 3. In this case, by regarding the speakers installed in the facility as the output device of the reader terminal, the content playback system of the present technology can seamlessly link the performance performed by the equipment in the facility with the audio output from the output device 21.

[0147] 13 , when user U1 enters a facility, the smartphone 3 carried by user U1 records a test sound output from a speaker 251 installed in the facility and measures the delay that occurs when the performance is performed as the playback delay of the reader terminal. Next, the smartphone 3 measures the playback delay of its own device and compensates for the playback delay based on the delay that occurs when the performance is performed and the playback delay of its own device.

[0148] In conjunction with the performance provided in the facility in response to a playback trigger transmitted from the trigger server 2, the smartphone 3 can output audio optimized for the individual user U1 or audio generated as stereophonic sound according to the position of the user U1 from the output device 21. The audio optimized for the individual user U1 includes audio corresponding to the language of the user U1, audio with content tailored to the hobbies and preferences of the user U1, audio according to the grade of the ticket purchased by the user U1, and the like.

[0149] - Coordination with drone shows and fireworks displays FIG. 14 is a diagram showing a second application example of the content playback system of the present technology.

[0150] Sound linked to a performance (reference content) such as a drone show or a fireworks display may be provided by the smartphone 3. In this case, by regarding the sound provided in the drone show or the sound of the fireworks display as a test sound, the content playback system of the present technology can link the performance such as the fireworks display with the sound output from the output device 21 of the smartphone 3 without any time lag.

[0151] 14 illustrates an example in which fireworks are linked to audio output from the output device 21 of the smartphone 3. As shown in Fig. 14, a speaker 261 is installed near the location (venue) where the fireworks are to be launched. The speaker 261 produces sound effects such as audio explaining the fireworks being launched and music that accompanies the fireworks.

[0152] The area near the fireworks launch site is an area where sound effects can be produced by the speaker 261. On the other hand, in areas far from the launch site, the fireworks can be seen, but the sound output from the speaker 261 does not reach those areas, so it can be said that it is difficult to produce sound effects in those areas.

[0153] The smartphones 3 carried by users U11 to U13 in an area where it is difficult to produce sound using the speaker 261 record the sound of fireworks as a test sound and measure the delay between when the fireworks are launched and when the sound of the fireworks reaches each user as the playback delay of the reader terminal. Next, the smartphones 3 measure the playback delay of their own devices and compensate for the playback delay based on the delay until the sound of the fireworks reaches each user and the playback delay of their own devices.

[0154] In conjunction with the fireworks (sounds), the smartphone 3 can output audio explaining the fireworks being launched, music that accompanies the fireworks, and the like from the output device 21. By using earphones, headphones, or the like, users U11 to U13 can experience the sound effects without worrying about the noise they cause to those around them.

[0155] Conventionally, the area where sound effects using speakers can be produced is limited, so there is a limit to the number of people who can experience the combined fireworks and sound effects. However, the content playback system of this technology can provide sound in perfect synchronization with the fireworks, making it possible to provide the combined fireworks and sound effects to many people, regardless of the distance from the fireworks launch site or the presence or absence of speakers 261.

[0156] Linkage with audio from concerts and events FIG. 15 is a diagram showing a third application example of the content playback system of the present technology.

[0157] Audio linked to audio (reference content) output from speakers installed at a concert venue, event venue, or the like may be provided by the smartphone 3. In this case, by regarding the speakers installed at the concert venue, or the like as output devices of the reader terminal, the content playback system of the present technology can link the audio output from the speakers and the audio output from the output device 21 of the smartphone 3 without any time lag.

[0158] 15, each user carries a smartphone 3, which records a test sound output from a speaker 281 installed near the stage in the concert venue, before the concert starts, and measures the delay that occurs when the speaker 281 outputs the sound as the playback delay of the reader terminal. Next, the smartphone 3 measures the playback delay of its own device.

[0159] After the concert starts, the smartphone 3 compensates for the playback delay based on the delay that occurs when the speaker 281 outputs sound and the playback delay of the smartphone 3. For example, the speaker of the smartphone 3 outputs the same sound as the sound output from the speaker 281.

[0160] The content playback system collects the location information of each smartphone 3 within the concert venue and outputs audio corresponding to the location information from the speaker of each smartphone 3, thereby enabling the speakers of the smartphone 3 to be used as multiple speakers.

[0161] Conventionally, at concerts and the like, the voices of performers and the sounds of musical instruments are output from speakers 281 installed near the stage, and it has only been possible to realize an acoustic effect in which sound images are localized on the stage. The content playback system of the present technology seamlessly links the sounds output from the speakers 281 and the speakers of the smartphone 3, thereby realizing an acoustic space in which many speakers are placed throughout the concert hall, and realizing an acoustic effect in which sound images are localized in various locations.

[0162] Specifically, the system will realize effects such as reducing the sense of distance between the audience and the performers, allowing the audience to hear the performers' voices from nearby, effects in which sound images move around the entire concert venue, and effects in which sounds are output from the audience seats in response to the fans' actions.

[0163] Linkage with Video Images FIG. 16 is a diagram showing a fourth application example of the content playback system of the present technology.

[0164] Audio linked to a moving image (reference content) displayed on the display device may be provided by the smartphone 3. In this case, by regarding a video player installed at an event venue or the like as a reader terminal, the content playback system of the present technology can seamlessly link the video (moving image) played by the video player and displayed on the display device with the audio output from the output device 21 of the smartphone 3.

[0165] The content playback system of FIG. 16 to which the present technology is applied is configured by connecting a display device 301 and smartphones 303A to 303C to a synchronization server 302.

[0166] The display device 301 repeatedly displays the video of the content and outputs audio corresponding to the video of the content. Here, the display device 301 is treated as a reader terminal. The display device 301 has a video player and a synchronization application. The video player performs playback processing of the content and displays the video of the content on the display unit of the display device 301. Furthermore, each time the video player performs playback processing of the content, it notifies the synchronization application that playback processing of the content has started.

[0167] The synchronization application notifies the synchronization server 302 of the playback start time when the video playback device starts the playback process of the content.

[0168] The synchronization server 302 notifies the smartphones 303A to 303C of the playback start time notified by the display device 301. The synchronization server 302 corresponds to the NTP server 1 and the trigger server 2 in FIG.

[0169] Smartphones 303A to 303C each have a synchronization app. The synchronization app measures the playback delay of display device 301 based on audio data recorded by a microphone of audio output from display device 301 and the playback start time notified by synchronization server 302. The synchronization app measures the playback delay of its own device, compensates for the playback delay of its own device based on the playback delay of display device 301 and the playback delay of its own device, and then outputs audio corresponding to the video displayed on display device 301 from an output device. Here, smartphones 303A to 303C correspond to smartphones 3A to 3C in FIG. 1 and are treated as follower terminals.

[0170] By compensating for the playback delay in the smartphones 303A to 303C, the video displayed on the display device 301 is synchronized with the audio output from the output devices of the smartphones 303A to 303C. Conventionally, if it was not possible to output audio corresponding to the video at a high volume at an event venue, it was difficult to deliver the audio to everyone at the venue. In the content playback system of the present technology, the audio corresponding to the video is output from earphones or the like connected to the smartphones 303A to 303C, so the audio can be delivered to everyone at the event venue.

[0171] Because audio is delivered to each person at the event venue, it is also possible to provide audio optimized for each individual user of smartphones 303A to 303C. Audio optimized for each individual user includes audio that is stereophonic based on the user's location, audio corresponding to the user's language, and audio guidance that corresponds to the user's level of understanding of the content.

[0172] In the following description, when there is no need to particularly distinguish between the smartphones 303A to 303C, they will simply be referred to as smartphones 303.

[0173] FIG. 17 is a block diagram showing a detailed configuration example of a content playback system in which moving images displayed on a display device and audio output from an output device of a smartphone are linked.

[0174] As shown in Figure 17, the content playback system is composed of the above-mentioned synchronization server 302 and smartphone 303, as well as a video playback device 321, a projector 322, an audio IF (Interface) 323, a speaker 324, a synchronization control device 325, a router 326, a router 331, and headphones 332.

[0175] The video player 321 corresponds to the video player on the display device 301 side described above. The video player 321 supplies a video signal representing the video of the content to the projector 322, and supplies an audio signal representing the audio of the content to the audio IF 323. A test signal (test sound) used to measure the playback delay of the display device 301 is embedded in the audio signal.

[0176] The projector 322 projects an image represented by the video signal supplied from the video player 321 .

[0177] The audio IF 323 supplies the audio signal supplied from the video player 321 to a speaker 324. The audio IF 323 detects that the video player 321 has started playing back the content, and notifies the synchronization control device 325 that the content playing back has started.

[0178] The synchronization control device 325 has the above-mentioned synchronization application on the display device 301 side. The synchronization control device 325 is connected to the synchronization server 302 via a router 326. The synchronization control device 325 notifies the synchronization server 302 of the playback start time at which the video playback device 321 starts the playback process of the content.

[0179] The synchronization server 302 has a server application. The synchronization server 302 is connected to the smartphone 303 via a router 331. The synchronization server 302 notifies the smartphone 303 of the playback start time notified by the display device 301.

[0180] The smartphone 303 measures the playback delay of the display device 301 based on audio data obtained by recording the audio output from the speaker 324 with a microphone and the playback start time notified by the synchronization server 302. The smartphone 303 measures the playback delay of its own device, compensates for the playback delay of its own device based on the playback delay of the display device 301 and the playback delay of its own device, and then outputs the audio of the content from the headphones 332.

[0181] For example, an event organizer may have smartphone 303 measure the playback delay of display device 301 and the playback delay of its own device in advance, and hand over smartphone 303 and headphones 332 to event participants when they enter the event venue. Smartphone 303 detects entry into the event venue using a Global Navigation Satellite System (GPS) signal, a beacon, a QR code (registered trademark), or the like, and acquires a playback timestamp of the video at the time of entry into the event venue. Smartphone 303 can provide users with audio linked to the video by seeking the playback position of the audio based on the playback timestamp of the video, taking into account the playback delay of display device 301 and its own device.

[0182] As described above, the content played by the follower terminal 3B (communication device) and the reference content played by the leader terminal are linked. The content linked to the reference content may be the same as the reference content. In this case, the same audio is output simultaneously from the output devices of the leader terminal 3A and the follower terminal 3B. The content linked to the reference content may also be content different from the reference content. In this case, the reference content and the audio output from the output device of the follower terminal 3B are linked without any delay.

[0183] <Regarding the Computer> The above-described series of processes can be executed by hardware or software. When the series of processes are executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into a general-purpose personal computer, etc.

[0184] FIG. 18 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0185] A CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 , and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

[0186] An input / output interface 505 is also connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc., and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. Also connected to the input / output interface 505 are a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives removable media 511.

[0187] In a computer configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program stored in the storage unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0188] The program executed by the CPU 501 is installed in the storage unit 508 by being recorded on, for example, a removable medium 511 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0189] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0190] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0191] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0192] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0193] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0194] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0195] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0196] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0197] (1) An information processing device comprising: a playback control unit that controls a start time at which a first playback device starts playback of content based on a measurement result of a first delay required from when a first playback device starts playback of the content until audio of the content is output from a first output device. (2) The information processing device described in (1), wherein the measurement of the first delay is performed based on sound recording data obtained by picking up a test sound output from the first output device with a microphone mounted on the first playback device. (3) The information processing device described in (2), wherein the sound recording data includes flag information indicating a start time at which the first playback device started playback of the test sound. (4) The information processing device described in (3), wherein the measurement result of the first delay includes a measurement result of the time from when the first playback device starts playback of the test sound to when the test sound is picked up by the microphone. (5) The information processing device according to any one of (1) to (4), wherein the playback control unit controls the start time of playback processing of the content based on a measurement result of the first delay and a measurement result of a second delay required from when a second playback device starts playback processing of a reference content linked to the content until audio of the reference content is output from a second output device. (6) The information processing device according to (5), wherein the measurement of the second delay is performed based on sound recording data obtained by picking up a test sound output from the second output device with a microphone mounted on the first playback device. (7) The information processing device according to (6), wherein the sound recording data includes flag information indicating a start time at which the second playback device started playback processing of the test sound. (8) The information processing device according to (7), wherein the measurement result of the second delay includes a measurement result of the time from when the second playback device starts playback processing of the test sound to when the test sound is picked up by the microphone. (9) The information processing device according to any one of (5) to (8), wherein the content and the reference content are the same content.(10) The information processing device according to any one of (5) to (8), wherein the reference content is content including a moving image, and the content is content including audio corresponding to the moving image. (11) The information processing device according to any one of (5) to (10), wherein the measurement result of the first delay indicates a relative delay based on the second delay. (12) The information processing device according to any one of (1) to (11), wherein the system time of the first playback device is synchronized with NTP time. (13) An information processing method comprising: controlling a start time at which the playback device starts playback processing of the content, based on a measurement result of a delay required from when the playback device starts playback processing of the content until audio of the content is output from an output device. (14) A program causing a computer to execute processing, including controlling a start time at which the playback device starts playback processing of the content, based on a measurement result of a delay required from when the playback device starts playback processing of the content until audio of the content is output from an output device.

[0198] 1 NTP server, 2 Trigger server, 3 Smartphone, 11 Network, 21 Output device, 71 Communication unit, 72 Control unit, 73 Memory unit, 74 Audio playback unit, 81 Playback control unit, 91 Audio buffer, 92 DA converter, 93 Audio output unit, 94 BT transmission unit, 171 Communication unit, 172 Control unit, 173 Memory unit, 174 Recording unit, 175 Audio playback unit, 181 Calculation unit, 182 Playback control unit, 191 Microphone, 192 AD converter, 193 Recording buffer, 194 Recording processing unit, 201 Audio buffer, 202 DA converter, 203 Audio output unit, 204 BT transmission unit

Claims

1. An information processing device having a playback control unit that controls the start time at which a first playback device starts playing back content based on the measurement result of a first delay from when the first playback device starts playing back content until the audio of the content is output from a first output device.

2. The information processing device according to claim 1, wherein the measurement of the first delay is performed based on recording data obtained by picking up the test sound output from the first output device with a microphone mounted on the first playback device.

3. The information processing device according to claim 2, wherein the recording data includes flag information indicating the start time at which the first playback device started the playback process of the test sound.

4. An information processing device as described in claim 3, wherein the measurement result of the first delay includes a measurement result of the time from the start time when the first playback device starts the playback process of the test sound to the time when the test sound is picked up by the microphone.

5. The information processing device according to claim 1, wherein the playback control unit controls the start time of the playback process of the content based on the measurement result of the first delay and the measurement result of the second delay required from when a second playback device starts playback processing of a reference content linked to the content until the audio of the reference content is output from a second output device.

6. An information processing device according to claim 5, wherein the second delay is measured based on recording data obtained by picking up a test sound output from the second output device with a microphone mounted on the first playback device.

7. The information processing device according to claim 6, wherein the recording data includes flag information indicating the start time at which the second playback device started the playback process of the test sound.

8. An information processing device as described in claim 7, wherein the measurement result of the second delay includes a measurement result of the time from the start time when the second playback device starts the playback process of the test sound to the time when the test sound is picked up by the microphone.

9. The information processing device according to claim 5, wherein the content and the reference content are the same content.

10. The information processing device according to claim 5, wherein the reference content is a content including a moving image, and the content is a content including audio corresponding to the moving image.

11. The information processing device according to claim 5, wherein the measurement result of the first delay indicates a relative delay based on the second delay.

12. The information processing device according to claim 1, wherein the system time of the first playback device is synchronized with NTP time.

13. An information processing method including controlling the start time at which a playback device starts playing back a content based on the measurement result of the delay between when the playback device starts playing back the content and when the audio of the content is output from an output device.

14. A program for causing a computer to execute a process including controlling the start time at which a playback device starts playback of a content based on the measurement result of the delay between when the playback device starts playback of the content and when the audio of the content is output from an output device.

Images