Signal processing device

Abstract

The present invention provides signal processing equipment and other devices capable of processing signals transmitted over different networks. [Solution] A signal processing device and the like as described in the claims are provided.

Description

Technical Field

[0001] The present invention relates to a signal processing device and the like.

Background Art

[0002] "MPEG-H MMT", which can transmit signals over multiple networks such as communication and broadcasting, may become a new standard.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Provided is a signal processing device and the like that can process signals transmitted via different networks.

Means for Solving the Problems

[0005] According to one aspect of the present invention, a signal processing device described in the claims is provided.

Brief Description of the Drawings

[0006] [Figure 1] A block diagram showing a schematic configuration of a distribution system according to the first embodiment. [Figure 2] A diagram schematically explaining the processing operation of the synchronization unit 23. [Figure 3] A block diagram showing a schematic configuration of a distribution system according to the second embodiment. [Figure 4] A block diagram showing a schematic configuration of a distribution system according to the third embodiment. [Figure 5] A block diagram showing a schematic configuration of a distribution system according to the fourth embodiment. [Figure 6]A block diagram showing the schematic configuration of the distribution system according to the fifth embodiment. [Figure 7] A block diagram showing the schematic configuration of the distribution system according to the sixth embodiment. [Figure 8] A diagram showing an example of the contents stored in memory unit 2E. [Figure 9] A block diagram showing the schematic configuration of the distribution system according to the seventh embodiment. [Figure 10] A block diagram showing the schematic configuration of the distribution system according to the eighth embodiment. [Figure 11] A diagram illustrating the interpolation process. [Figure 12] A block diagram showing the schematic configuration of the recorder 5 according to the ninth embodiment. [Figure 13] A block diagram showing the schematic configuration of a smartphone 6 according to the tenth embodiment. [Figure 14] A diagram showing an example of a two-dimensional display. [Modes for carrying out the invention]

[0007] (First embodiment) When video and audio signals are transmitted over different networks, transmission delays can occur. However, if timing information such as presentation time information and timestamps is added to both the video and audio signals, the receiver can synchronize them for playback. However, if the equipment playing the video signal and the equipment playing the audio signal are different, a typical example being when the video signal is played on a television and the audio signal is played on external speakers using an amplifier connected to the television, synchronization becomes difficult.

[0008] Therefore, in the first embodiment, a mechanism is provided that allows the video signal and the audio signal to be played back in synchronization even in such cases.

[0009] Figure 1 is a block diagram illustrating the schematic configuration of a distribution system according to the first embodiment. This distribution system consists of a broadcasting device 1, a television (signal processing device) 2, an amplifier 3 connected to the television 2, and a speaker 4 connected to the amplifier 3. Note that the amplifier 3 and speaker 4 may be integrated. Furthermore, the television 2 and amplifier 3 constitute a signal processing system.

[0010] The broadcasting device 1 includes a content storage unit 11, a video transmission unit 12, and an audio transmission unit 13. However, the content storage unit 11, the video transmission unit 12, and the audio transmission unit 13 do not necessarily have to be located within a single broadcasting device 1, and may be distributed across multiple devices.

[0011] The content storage unit 11 stores the content signal. The content signal consists of a video signal and an audio signal that are associated with each other and should be played back in sync. The video transmission unit 12 transmits the video signal to the television via a broadcast network. The audio transmission unit 13 transmits the audio signal to the television via a communication network (such as the Internet; the same applies hereinafter).

[0012] Timing information is attached to the video and audio signals. For example, the video and audio signals consist of multiple packets, and each packet may contain timing information. The timing information may be presentation time information, and the receiver can understand which video and audio packets should be played back at each time. Alternatively, the timing information may be a timestamp, and the receiver can simultaneously play back a video packet with a certain timestamp and an audio packet with the same timestamp.

[0013] Television 2, an example of a signal processing device, includes a video receiving unit 21, an audio receiving unit 22, a synchronization unit 23, a display 24, a speaker 25, and an output unit 26. The video receiving unit 21 receives a video signal via a broadcast network. The audio receiving unit 22 receives an audio signal via a communication network. However, the received video signal and audio signal are not necessarily synchronized, and one of them may be received earlier due to, for example, the difference in transmission delay between the broadcast network and the communication network.

[0014] Therefore, the synchronization unit 23 controls the playback timing of the video signal and / or the audio signal so that the video signal and the audio signal are played back synchronously. More specifically, the synchronization unit 23 synchronizes the video signal and the audio signal based on timing information. For example, when the transmission delay time varies depending on the network, the synchronization unit 23 compensates for the difference in transmission delay time by delaying the signal received earlier.

[0015] FIG. 2 is a diagram schematically explaining the processing operation of the synchronization unit 23. As shown in FIG. 2(a), assume that the audio signal is received dt1 later than the video signal. This can be grasped from the timing information added to the video signal and the audio signal. In this case, the synchronization unit 23 delays the video signal by dt1 by using, for example, a frame memory, and synchronizes it with the audio signal as shown in FIG. 2(c). On the other hand, as shown in FIG. 2(b), when the video signal is received dt2 later than the audio signal, the synchronization unit 23 delays the audio signal by dt2 and synchronizes it with the video signal as shown in FIG. 2(c).

[0016] The synchronization unit 23 can perform such processing in units to which timing information is added. For example, when timing information is added to each packet, the synchronization unit 23 adjusts the delay time for each packet. Also, when timing information is added to each frame, the synchronization unit 23 adjusts the delay time for each frame.

[0017] In this way, synchronized video and audio signals are generated. The former can be played back on the display 24 built into the television 2. The latter can be played back on the speaker 25 built into the television 2, but if the audio signal is played back on an external speaker 4 via the amplifier 3, the speaker 25 of the television 2 may be muted. In addition, video processing and / or audio processing may be performed before and after processing in the synchronization unit 23 as needed.

[0018] Television 2 is connected to amplifier 3, for example, by an HDMI® cable. When playing an audio signal through speaker 4 connected to amplifier 3, the output unit 26 outputs an audio signal from the synchronization unit 23, in other words, an audio signal synchronized with the video signal, to amplifier 3. The audio receiving unit 31 of amplifier 3 receives this audio signal. This audio signal is then processed as needed and played back by speaker 4.

[0019] In television 2, if the time from when the synchronized video signal is output from the synchronization unit 23 until it is played on the display 24 is approximately equal to the time from when the synchronized audio signal is output from the output unit 26 to the audio receiving unit 31 of amplifier 3 until it is played on speaker 4, then the video signal played on television 2 and the audio signal played on speaker 4 via amplifier 3 can be synchronized. If television 2 and amplifier 3 are manufactured by the same company, and the time required for internal processing in amplifier 3 is known, the synchronization unit 23 can synchronize and play the video signal and audio signal with higher precision by taking this time into consideration.

[0020] Thus, in the first embodiment, the video signal and the audio signal are synchronized before the audio signal is transmitted to the amplifier 3. Therefore, even if the television 2 that plays the video signal and the amplifier 3 (speaker 4) that plays the audio signal are different, both signals can be played in sync. With this configuration, the amplifier 3 does not need a communication function for receiving the audio signal via a communication network or a mechanism for adjusting the playback timing of the audio signal.

[0021] (Second embodiment) The second embodiment is a modification of the first embodiment, in which the audio signal before synchronization is output to the amplifier, and the information necessary for the synchronization process is also notified to the amplifier, and the synchronization process is performed on the amplifier side. The differences will be explained below.

[0022] Figure 3 is a block diagram illustrating the schematic configuration of a distribution system according to the second embodiment. Note that the broadcasting equipment shown in Figure 1 is omitted. The television 2 further includes a control unit 27.

[0023] The control unit 27 generates a signal (hereinafter referred to as a synchronization control signal) that can be used to synchronize the video signal and the audio signal. Specifically, the control unit 27 may detect the delay time (transmission time difference) between the video signal and the audio signal and generate a synchronization control signal that indicates this delay time. The delay time here refers to the time that indicates how much later each of the video signal and the audio signal is received relative to the other, for example, dt1 in Figure 2(a) and dt2 in Figure 2(b). The delay time is determined from timing information added to the video signal and the audio signal. Note that even if a signal is intended for other purposes, if it can be used to synchronize the video signal and the audio signal, that signal may be used as a synchronization control signal.

[0024] The output unit 26 outputs the audio signal before synchronization and the synchronization control signal to the amplifier 3. These two signals may both be supplied to the amplifier 3 via an HDMI cable, for example. Alternatively, the audio signal may be supplied to the amplifier 3 via an HDMI cable, and the synchronization control signal may be supplied to the amplifier 3 via a separate path, such as short-range wireless communication.

[0025] The audio receiving unit 31 of amplifier 3 receives the audio signal before it is synchronized. Amplifier 3 also has a synchronization unit 32, which, based on a synchronization control signal, synchronizes the audio signal received by the audio receiving unit 31 with the video signal played on television 2. If the synchronization control signal indicates a delay time, the synchronization unit 32 delays the audio signal according to that delay time. For example, as shown in Figure 2(b), if the video signal is delayed by dt2, the synchronization unit 32 delays the audio signal by dt2 and synchronizes it with the video signal on television 2. The synchronized audio signal is then played back by speaker 4.

[0026] As shown in Figure 2(a), if the audio signal is delayed, the TV's synchronization unit 23 delays the video signal to synchronize the video signal and the audio signal. Therefore, the amplifier's synchronization unit 32 does not need to delay the audio signal. Thus, the synchronization control signal may indicate that there is no need to delay the audio signal, or it may simply indicate that the audio signal is delayed compared to the video signal, or it may indicate that the delay time is 0.

[0027] Thus, in the second embodiment, since a synchronization control signal is output to the amplifier 3, both signals can be synchronized and reproduced even if the television 2 that reproduces the video signal and the amplifier 3 that reproduces the audio signal are different. With this configuration, the amplifier 3 does not need a communication function to receive the audio signal via a communication network. Furthermore, since the audio signal is supplied to the amplifier 3 from near the audio receiving unit 22 of the television 2, this configuration is suitable when it is desirable to perform audio processing on the amplifier 3 side as much as possible, while avoiding audio processing on the television 2 side. The synchronization control signal may also be generated when the synchronization unit 23 performs the synchronization process.

[0028] (Third embodiment) The third embodiment allows switching between supplying the amplifier with the audio signal before synchronization and the synchronization control signal, or supplying the amplifier with the synchronized audio signal. The following description will focus on the differences from the first and second embodiments.

[0029] Figure 4 is a block diagram illustrating the schematic configuration of a distribution system according to the third embodiment. Note that the broadcasting equipment shown in Figure 1 is omitted. The television 2 further includes an information acquisition unit 28 that acquires information from the amplifier 3.

[0030] The information for amplifier 3 indicates whether amplifier 3 should receive the audio signal before synchronization and the synchronization control signal, or the synchronized audio signal. Alternatively, the information for amplifier 3 may indicate whether amplifier 3 is capable of processing the synchronization control signal. For example, the information for amplifier 3 may be information that can determine whether amplifier 3 has the synchronization unit 32 shown in Figure 3. In this case, the information for amplifier 3 may be information that directly indicates whether or not it has the synchronization unit 32, or it may be identification information such as the model number of amplifier 3. In the latter case, the model numbers of amplifier 3 with the synchronization unit 32 and the model numbers of amplifier 3 without the synchronization unit 32 can be stored in a database in (or accessible from) television 2. As another example, the information for amplifier 3 may be set by the user for amplifier 3.

[0031] The information acquisition unit 28 may acquire information from the amplifier 3, for example, via an HDMI cable, or it may accept settings directly from the user. The information from the amplifier 3 is notified to the output unit 26.

[0032] In addition, user settings are input to the output unit 26. These user settings determine whether to supply a synchronized audio signal to the amplifier 3, as shown in Figure 1, or to supply the unsynchronized audio signal and the synchronization control signal to the amplifier 3, as shown in Figure 3.

[0033] The output unit 26 then outputs either the audio signal and synchronization control signal before synchronization, or the synchronized (post-synchronization) audio signal, to the amplifier 3 based on the information from the amplifier 3 and / or user settings. In the latter case, the synchronization control signal may or may not be supplied to the amplifier 3.

[0034] When an audio signal and a synchronization control signal are supplied before synchronization, the synchronization unit (not shown) of amplifier 3 synchronizes the video signal and the audio signal by delaying the audio signal based on the synchronization control signal. When a synchronized audio signal is supplied, the synchronization unit does not need to operate, even if one exists. It is also desirable to notify amplifier 3 from the output unit 26 of whether the audio signal is before or after synchronization.

[0035] The relationship between the information from amplifier 3 and user settings can be prioritized as appropriate. For example, the user could be given three options as user settings: "Before Synchronization," "After Synchronization," and "Automatic." If "Before Synchronization" or "After Synchronization" is selected, the selected audio signal would be supplied regardless of the information from amplifier 3. If "Automatic" is selected, the audio signal would be supplied based on the information from amplifier 3. Another example is to prioritize the information from amplifier 3 as a general rule, and if the information from amplifier 3 is unknown (cannot be obtained), the audio signal according to the user setting would be supplied.

[0036] Furthermore, if the output unit 26 uses only user settings, the information acquisition unit 28 can be omitted, and if the output unit 26 uses only information from the amplifier 3, the user settings can be omitted.

[0037] Furthermore, the first to third embodiments described above involved receiving video and audio signals on a television and outputting the audio signal to an amplifier. In contrast, in a signal processing device that receives video and audio signals, where the audio signal is reproduced by the signal processing device and the video signal is reproduced by another connected device, the video signal is output to the other device. In this case as well, the synchronized video signal may be output as described in the first embodiment, or the unsynchronized video signal and synchronization control signal may be output as described in the second embodiment, or the user may be able to select either of the two as described in the third embodiment.

[0038] Furthermore, in the first to third embodiments, when a video signal is played back on an external device connected to the television 2, the output unit 26 may output the video signal to the external device. In this case as well, the synchronized video signal may be output as described in the first embodiment, the unsynchronized video signal and synchronization control signal may be output as described in the second embodiment, or either of the two may be selectable as described in the third embodiment.

[0039] (Fourth embodiment) Paragraph 0053 of Patent Document 1 discloses that after selecting a channel via a broadcast stream, a separate instruction to acquire a communication stream is issued in response to a viewing instruction from the user. However, in this case, the user must separately select a channel related to the broadcast network and issue a viewing instruction related to the communication network, which is very cumbersome. Therefore, in this embodiment, a mechanism is provided that allows users to easily configure settings for both networks.

[0040] Figure 5 is a block diagram illustrating the schematic configuration of a distribution system according to the fourth embodiment. The distribution system consists of a broadcasting device 1 and a television 2.

[0041] The broadcasting device 1 includes a content storage unit 11, a broadcast transmission unit 14, and a communication unit 15. However, the content storage unit 11, the broadcast transmission unit 14, and the communication unit 15 do not necessarily have to be located within a single broadcasting device 1, and may be distributed across multiple devices.

[0042] The content storage unit 11 stores multiple content signals. Each content signal consists of a video signal and an audio signal that are associated with each other and should be played back in sync, and timing information is attached to each of them.

[0043] The broadcast transmission unit 14 multiplexes the video signals from multiple content signals and transmits the video signals to the television 2 via the broadcast network. In this embodiment, the communication unit 15 is assumed to transmit audio signals unicast, and receives a request signal (described later) from the television 2 via the communication network, and acquires the audio signal indicated by the request signal from the content storage unit 11 and transmits it to the television 2 via the communication network.

[0044] Television 2, an example of a signal processing device, includes an operation reception unit 29, a broadcast reception unit 2A, a tuner (selection unit) 2B, a request unit 2C, a communication unit 2D, a synchronization unit 23, a display 24, and a speaker 25.

[0045] The operation reception unit 29 receives user operations. User operations may be operations on the remote control or operations on buttons (not shown) provided on the TV 2 main unit. In this embodiment, the user operation is to set which channel from the multiplexed video signal to select. This selection is performed by pressing a channel selection key on the remote control, selecting currently broadcasting content from the electronic program guide displayed on the display 24, or pressing a channel selection button provided on the TV 2 main unit.

[0046] The broadcast receiving unit 2A receives multiplexed video signals via the broadcast network. The tuner 2B selects one of the multiplexed video signals in response to a user operation received by the operation receiving unit 29.

[0047] The request unit 2C generates a request signal to request an audio signal associated with the selected video signal. In this embodiment, the request unit 2C generates a request signal in response to a user operation received by the operation reception unit 29 from the user.

[0048] To give a simple example, suppose there are content signals from the 1st to 3rd broadcasting stations (hereinafter referred to as 1ch, 2ch, and 3ch, respectively). When the user selects 2ch through user operation, tuner 2B selects the video signal for 2ch, and request unit 2C generates a request signal to request the audio signal for 2ch.

[0049] Thus, a single user operation (such as pressing a channel selection key on the remote control as described above) is used for both channel selection by the tuner 2B and the generation of a request signal by the request unit 2C. This allows the user to configure settings for both the broadcast network and the communication network with a single operation, saving time and effort.

[0050] Communication unit 2D transmits a request signal to broadcasting device 1 via the communication network. Communication unit 2D then receives an audio signal transmitted from communication unit 15 in response to this request signal. The received audio signal is an audio signal associated with the video signal selected by tuner 2B.

[0051] The synchronization unit 23, display 24, and speaker 25 are as described above. It is desirable that the synchronization unit 23 synchronizes the video signal selected by the tuner 2B with the audio signal received by the communication unit 2D, taking into account the delay from when the request unit 2C generates a request signal, when the communication unit 2D transmits this request signal to the broadcasting device 1, until when the communication unit 2D receives the audio signal.

[0052] Thus, in the fourth embodiment, both control over the broadcast network and control over the communication network can be performed with a single user operation. This reduces the effort required from the user.

[0053] Furthermore, if the signal processing device is a smartphone or tablet, user operation may be via a touch panel, or if it is a personal computer, via a mouse or keyboard; any input interface that allows for channel selection, etc., is acceptable. This is also true for other embodiments. In addition, the signals transmitted over the broadcast network and communication network do not necessarily have to be video signals and audio signals; both may be video signals (or audio signals), or they may be some kind of control signal.

[0054] (Fifth embodiment) The fifth embodiment, which will be described next, is a modification of the fourth embodiment. The following explanation will focus on the differences from the fourth embodiment.

[0055] Figure 6 is a block diagram illustrating the schematic configuration of a distribution system according to the fifth embodiment. The distribution system consists of a broadcasting device 1 and a television 2. In contrast to the fourth embodiment, the request unit 2C in the television 2 receives notification from the tuner 2B that the tuner 2B has selected a video signal and generates a request signal.

[0056] To give an example similar to that of the fifth embodiment, when channel 2 is selected by user operation, tuner 2B selects the video signal for channel 2. Then, tuner 2B notifies request unit 2C that it has selected the video signal for channel 2. In response, request unit 2C generates a request signal to request the audio signal for channel 2.

[0057] This also reduces the effort required from the user. In both the fourth and fifth embodiments, settings for both the broadcast network and the communication network can be configured based on a single user operation.

[0058] The roles of tuner 2B and request unit 2C may be swapped. That is, request unit 2C may generate a request signal that requests one audio signal based on user operation, and notify tuner 2B which audio signal was requested, and tuner 2B may perform station selection upon receiving this notification.

[0059] (Sixth Embodiment) The sixth embodiment is a modification of the fourth and fifth embodiments. In this embodiment, it is assumed that in a single content signal, multiple audio signals are associated with a single video signal. The multiple audio signals may be, for example, a main audio and a secondary audio, multiple audios in different languages, or audios with different amounts of information, such as stereo audio and 5.1ch audio.

[0060] Figure 7 is a block diagram showing the schematic configuration of the distribution system according to the sixth embodiment. A difference from the fourth and fifth embodiments is that the television 2 further includes a storage unit 2E. The storage unit 2E stores which of the above-mentioned audio signals should be requested. Note that this storage unit 2E may be located outside the television.

[0061] Figure 8 shows an example of the contents stored in memory unit 2E. In this example, for each video signal (each channel), it is pre-stored which of the multiple audio signals is to be requested. The stored contents may be set by the user according to their preferences or language of use. Alternatively, they may be set automatically according to a contract, for example, if a paid contract to receive 5.1ch audio is made, this may be set via the broadcast network or communication network.

[0062] Each item of memory content may be common to all channels, or it may differ from channel to channel. For example, it is desirable that language be common to all channels. Also, it is desirable that the amount of information differs from channel to channel, for example, by using 5.1ch audio only for channels that primarily broadcast music programs.

[0063] The request unit 2C generates a request signal to request a specific audio signal based on the user operation received by the operation reception unit 29 and the contents stored in the storage unit 2E. In the example in Figure 8, if channel 2 is selected by the user operation, the request unit 2C refers to the storage unit 2E and generates a request signal to request Japanese and stereo audio from among the 2ch audio signals, and transmits it to the broadcasting device 1.

[0064] The communication unit 15 of the broadcasting device 1, when there are multiple audio signals associated with a 2ch video signal, selects one of them according to the requested signal and transmits it to the television via the communication network.

[0065] Thus, in the sixth embodiment, since the required audio signals are stored in the memory unit 2E, the user's effort can be reduced even when there are multiple audio signals associated with the video signal.

[0066] In Figure 7, the request unit 2C operates based on user operations received by the operation reception unit 29, similar to the fourth embodiment. However, the request unit 2C may also operate based on notifications from the tuner 2B, similar to the fifth embodiment.

[0067] (Seventh Embodiment) In embodiments 4 through 6, audio signals were transmitted via a communication network in response to a request from television 2. In contrast, embodiment 7, which will be described next, transmits multiple audio signals using so-called multicast, and television 2 selects one of the audio signals. The differences will be explained below.

[0068] Figure 9 is a block diagram illustrating the schematic configuration of a distribution system according to the seventh embodiment. In this embodiment, the communication unit 15 of the broadcasting device 1 transmits multiple audio signals (multiplexed as necessary) without receiving a request from the television 2, without specifying a destination. This type of transmission is also called multicast and is similar to transmission via a broadcasting network. Multiple audio signals are, for example, audio signals in content signals from multiple broadcasting stations.

[0069] The communication unit 2D in television 2 receives multiple audio signals. Television 2 also has a selection unit 2F, which selects one of the audio signals according to the user operation received by the operation reception unit 29.

[0070] In this embodiment, a single user operation is used for both channel selection by the tuner 2B and selection by the selection unit 2F. This allows the user to configure settings for both the broadcast network and the communication network with a single operation, saving time and effort.

[0071] Although this embodiment is similar to the fourth embodiment, it may also be similar to the fifth embodiment. That is, the tuner 2B may select one video signal based on user operation and notify the selection unit 2F of which audio signal has been selected, and the selection unit 2F may make a selection upon receiving this notification. Of course, the roles of the tuner 2B and the selection unit 2F may also be swapped. Furthermore, similar to the sixth embodiment, a storage unit 2E may be provided, and the selection unit 2F may also refer to the storage unit 2E when making a selection.

[0072] (Eighth embodiment) In the case of smartphones and other devices, receiving signals from broadcast networks or communication networks can be difficult. Therefore, it is necessary to be able to reproduce video and audio signals as much as possible even in such situations. Therefore, in the eighth embodiment, audio signals are received from both the broadcast network and the communication network, and one of them is selected for playback.

[0073] Figure 10 is a block diagram showing the schematic configuration of the distribution system according to the eighth embodiment. The differences from the above embodiment will be explained in detail. In this embodiment, a smartphone is assumed as the signal processing device, but it can be applied to other electronic devices as well, and is particularly suitable for mobile devices such as notebook PCs and tablets, and mobile communication devices such as in-car televisions (which can also be built into car navigation systems).

[0074] The broadcast transmission unit 14 in the broadcasting device 1 transmits a video signal and a first audio signal to be reproduced in synchronization with it via the broadcasting network. The communication unit 15 transmits a second audio signal to be reproduced in synchronization with the video signal via the communication network. The second audio signal may be substantially the same as the first audio signal (except for differences due to differences in the transmission network) or it may be different; specific examples will be described later.

[0075] The smartphone 6 includes a broadcast receiving unit 61, a communication receiving unit 62, a determination unit 63, an output unit 64, a synchronization unit 65, a display 66, and a speaker 67. The broadcast receiving unit 61 receives a video signal and a first audio signal via the broadcast network. The communication receiving unit 62 receives a second audio signal via the communication network. The determination unit 63 determines whether to play the first audio signal or the second audio signal. The output unit 64 selects and outputs either the first audio signal or the second audio signal according to the determination result. The output unit 64 may perform predetermined processing on the first or second audio signal before outputting it for subsequent processing.

[0076] The synchronization unit 65 and other components are as described above. Since the video signal and the first audio signal should be synchronized, the synchronization unit 65 may synchronize the video signal and the second audio signal only when the second audio signal is output.

[0077] The determination unit 63 may determine which of the two audio signals is superior to reproduce based on the status of the broadcast network and the communication network, and the reception quality of the first and second audio signals. The status of the broadcast network and the communication network may be determined, for example, from the reception strength of the broadcast network (broadcast waves) or the reception strength of the communication network (communication radio waves). Reception quality may be determined from the error rate and S / N ratio detected when demodulating and decoding the video signal and audio signal. In any case, the relationship between the network status, the reception quality of the audio signal, and which one should be selected should be predetermined.

[0078] If the first audio signal and the second audio signal differ only due to differences in the transmission network, and are substantially the same as the audio being reproduced, the determination unit 63 may determine that whichever of the first and second audio signals can be reproduced with higher quality should be reproduced. If both signals can be reproduced to a similar degree, the determination unit 63 may determine that either one should be reproduced.

[0079] The first audio signal may be stereo audio, while the second audio signal may be 5.1ch audio, meaning the second audio signal may be of higher quality. Furthermore, the first audio signal may be in the base language (for example, Japanese in the case of a smartphone intended for Japanese users), while the second audio signal may be in a language selected by the user from among multiple languages.

[0080] In such cases, the user is likely to prefer the playback of the second audio signal over the first audio signal, so the determination unit 63 should determine that the second audio signal should be prioritized for playback. For example, if both the first and second audio signals can be played to the same degree, the determination unit 63 should determine that the second audio signal should be played. Only if playback of the second audio signal is impossible (including cases where playback is unbearable or where the first audio signal sounds clearer) should the first audio signal be played.

[0081] Conversely, if it is determined that the user desires the playback of the first audio signal, the determination unit 63 may determine that the first audio signal should be played preferentially. The details are the same as described above.

[0082] Incidentally, even if it is determined that the second audio signal should be played, for example, a portion of the second audio signal may be missing. In that case, the output unit 64 may output the missing portion in real time by supplementing it with the corresponding portion of the first audio signal.

[0083] Figure 11 is a schematic diagram illustrating the interpolation process. If packet P2 of the second audio signal that the communication receiving unit 62 is supposed to receive cannot be received properly, the output unit 64 performs interpolation using packet P12 (corresponding to packet P2) of the first audio signal received by the broadcast receiving unit 61. In this case, the output unit 64 can determine the packet in the first audio signal that corresponds to the packet in the second audio signal that could not be received properly, based on the timing information contained in the first and second audio signals. That is, packet P2 has a timing signal T2 attached to it, and it can be seen that packet P12, which has the corresponding timing signal T2 attached, should be used for interpolation. If necessary, packets before and after packet P12 may also be used for interpolation.

[0084] If packet P12 and packet P2 are substantially identical, the output unit 64 simply replaces the portion of packet P2 with packet P12. If they are different, the output unit 64 may perform the necessary conversions. For example, if the first audio signal is stereo and the second audio signal is 5.1ch audio, the output unit 64 may convert the stereo audio to 5.1ch audio using a known method during interpolation. If the former and latter are in different languages, the output unit 64 may translate them during interpolation. Furthermore, if they are not audio signals but video signals with different resolutions, the output unit 64 may resize them during interpolation.

[0085] Thus, in the eighth embodiment, since a determination unit 63 is provided, the appropriate audio signal can be reproduced from the audio signal received via the broadcast network and the audio signal received via the communication network.

[0086] In this embodiment, the output destination of the output unit 64 is ultimately the speaker 67. However, if the signal processing device is a television, the output destination may be an amplifier as described in the first embodiment, or if the signal processing device is a recorder, the output destination may be a storage medium. There are no particular restrictions on the output destination.

[0087] (Ninth embodiment) In this embodiment, a recorder is primarily assumed as the signal processing device. Video and audio signals containing timing information have a larger data volume than video and audio signals without timing information, thus putting a strain on the recording capacity. Therefore, in this embodiment, the timing information is removed before recording.

[0088] Figure 12 is a block diagram illustrating the schematic configuration of a recorder 5 according to the ninth embodiment. The recorder 5, an example of a signal processing device, comprises a video receiving unit 51, an audio receiving unit 52, a synchronization unit 53, an output unit 54, a recording medium 55, and a reading unit 56. The recording medium 55 may be located inside the recorder 5, such as an internal hard disk, or it may be inserted from an external source, such as an optical disc.

[0089] The video receiving unit 51 receives the video signal, and the audio receiving unit 52 receives the audio signal, via different networks (for example, the former via a broadcasting network and the latter via a communication network). The synchronization unit 53 synchronizes the video signal and the audio signal based on the timing information contained in the video signal and the audio signal. The output unit 54 outputs the synchronized video signal and audio signal to the recording medium 55, and records the video signal and audio signal in a synchronized state thereon.

[0090] Here, the video signal and audio signal may be recorded on the recording medium with timing information included, but it is preferable for the output unit 54 to remove at least a portion of the timing information in the video signal and / or the timing information in the audio signal before outputting. Timing information is necessary to synchronize the video signal and audio signal, but it is not necessarily required to retain it once they have been synchronized. Therefore, by removing the timing information, the recording can be done efficiently on the recording medium 55.

[0091] Furthermore, the output unit 54 may convert the format of the video signal and audio signal and record them on the recording medium 55. For example, it may convert a signal compliant with MPEG-H MMT to a signal compliant with MPEG2-TS and record it.

[0092] The reading unit 56 reads the video and audio signals recorded on the recording medium 55 in a synchronized manner and outputs them to an external device (not shown), such as a television. If necessary, the reading unit 56 may convert the format of the read video and audio signals, such as by adding timing information, before outputting them. This is effective when the external device to which the output is received is expected to process video and audio signals that include timing information, or to process signals in a specific format.

[0093] Thus, in the ninth embodiment, the timing signal is removed during recording, which reduces the amount of data required for recording. The output unit 54 may output the video signal and audio signal, from which timing information has been removed, to a device other than the recording medium 55 (for example, in the first to third embodiments). Furthermore, the synchronization unit 53 may remove timing information when synchronizing the video signal and audio signal.

[0094] (Tenth embodiment) When receiving signals from both a broadcast network and a communication network, it is insufficient if only the broadcast network or only the communication network is received correctly. In particular, with mobile devices such as smartphones, stable reception of signals from the broadcast network and communication network is not possible if the device is not in an appropriate location. Therefore, this embodiment provides a mechanism to assist the user in determining an appropriate location.

[0095] Figure 13 is a block diagram showing the schematic configuration of a smartphone 6 according to the tenth embodiment. In this embodiment, the smartphone 6, which is an example of a signal processing device, includes a broadcast reception level detection unit 68, a communication reception level detection unit 69, and a display control unit 6A. The rest is the same as in the embodiments described above.

[0096] The broadcast reception level detection unit 68 detects the reception level of the broadcast network (broadcast waves). The communication reception level detection unit 69 detects the reception level of the communication network (communication radio waves). The display control unit 6A displays the reception strength of the broadcast network and the reception strength of the communication network in two dimensions on the display 66.

[0097] Figure 14 shows an example of a two-dimensional display. As shown in the figure, the reception level of the broadcast network can be displayed on the horizontal axis, and the reception level of the communication network can be displayed on the vertical axis.

[0098] The circles in the figure indicate the current signal strength (with a slight time lag), and if the signal strength changes due to the movement of the smartphone 6, it is desirable for the display control unit 6A to move the position of these circles in real time and display the result.

[0099] The square marks in the diagram indicate target values ​​for properly reproducing the currently received broadcast and communication signals. These positions may vary depending on the type of broadcast signal and the selected channel. There may be multiple target values, including a target value for the highest quality reproduction and a target value for the minimum quality reproduction.

[0100] The user can then use this display to adjust their position or antenna so that both the current broadcast network signal strength and the communication network signal strength exceed the target signal strength.

[0101] Such a display is initiated, for example, in response to user action. The display control unit 6A may also display the maximum received signal strength of the broadcast network and communication network from the start point onward, and in that case, its position may be moved each time the maximum signal strength is updated. This allows the user to understand the received signal strength to the extent possible at that time, even if the received signal strength does not reach the target value.

[0102] All or part of the signal processing device described above may be functions realized by a processor executing a predetermined program. Furthermore, not all functional units in each signal processing device are essential; they may be omitted as needed, other functional units may be added, or they may be distributed across multiple devices. Moreover, multiple embodiments may be combined. Based on the above embodiments, for example, the following configuration can be conceived.

[0103] [A-1] (First Embodiment) A first receiving unit that receives a first signal via a first network, A second receiving unit receives a second signal, which is to be reproduced in synchronization with the first signal, via a second network. A synchronization unit that synchronizes the first signal and the second signal (controls the first signal and / or the second signal so that the first signal and the second signal are reproduced in sync), A signal processing device comprising: an output unit that outputs the synchronized second signal to an external device.

[0104] [A-1'] A first receiving unit that receives video signals via a first network, A second receiving unit receives an audio signal to be played back in synchronization with the aforementioned video signal via a second network, A synchronization unit that synchronizes the video signal and the audio signal, A display on which the aforementioned video signal is reproduced, It comprises an output section that outputs the synchronized audio signal to an amplifier, A television in which one of the first network and the second network is a broadcasting network and the other is a communications network.

[0105] [B-1] (Second Embodiment) A first receiving unit that receives a first signal via a first network, A second receiving unit receives a second signal, which is to be reproduced in synchronization with the first signal, via a second network. A control unit that generates control signals that can be used to synchronize the first signal and the second signal, A signal processing device comprising an output unit that outputs the control signal and the second signal to an external device.

[0106] [C-1] (Third Embodiment) A first receiving unit that receives a first signal via a first network, A second receiving unit receives a second signal, which is to be reproduced in synchronization with the first signal, via a second network. A control unit that generates control signals that can be used to synchronize the first signal and the second signal, A synchronization unit that synchronizes the first signal and the second signal, A signal processing device comprising: an output unit that outputs either the control signal and the second signal, or a synchronized second signal, to the external device based on information of the connected external device and / or user settings.

[0107] [C-2] The aforementioned control signal is This indicates the delay time difference between the first signal and the second signal, The time to delay the second signal is indicated by: A signal processing device as described in B-1 or C-1.

[0108] [D-1] A receiving device comprising a receiving unit that receives the second signal output from the output unit of the signal processing device described in any of A-1 to C-2.

[0109] [E-1] A signal processing device described in any of A-1 to C-2, A signal processing system comprising the receiving device described in D-1.

[0110] [F-1] (Fourth embodiment) A receiving unit that receives signals containing multiple signals (multiplexed multiple signals) via a broadcast network, A selection unit that selects a first signal from the plurality of signals, A request unit generates a request signal requesting that a second signal, which should be reproduced in synchronization with the first signal, be transmitted via a communication network. It includes an operation reception unit that receives user input, A signal processing device wherein, based on a single user operation, the selection unit not only selects the first signal, but the request unit also generates the request signal.

[0111] [F-2] The aforementioned user operation is a station selection operation, as described in the signal processing device F-1.

[0112] [F-3] (Sixth Embodiment) The signal processing apparatus according to F-1,2, wherein there are multiple signals to be reproduced in synchronization with the first signal, and the request unit generates the request signal by referring to a storage unit that stores which of these signals is requested as the second signal.

[0113] [G-1] A first receiving unit that receives a signal containing multiple signals via a first network, A selection unit that selects a first signal from the plurality of signals, A second receiving unit receives a second signal, which is to be reproduced in synchronization with the first signal, via a second network. It includes an operation reception unit that receives user input, A signal processing device in which, based on one user operation, the first signal is selected and the second signal is received.

[0114] [H-1] (Seventh Embodiment) A first receiving unit that receives a signal containing multiple signals via a first network, A first selection unit that selects a first signal from the plurality of signals, A second receiving unit that receives a signal containing multiple signals via a second network, A second selection unit selects a second signal from the plurality of signals to be reproduced in synchronization with the first signal, It includes an operation reception unit that receives user input, A signal processing device in which, based on one user operation, the first selection unit not only selects the first signal, but the second selection unit also selects the second signal.

[0115] [I-1] (Eighth embodiment) A first receiving unit that receives a video signal and a first audio signal to be played back in sync therewith via a first network, A second receiving unit receives a second audio signal, which is to be played back in synchronization with the aforementioned video signal, via a second network. A determination unit that determines which of the first audio signal and the second audio signal should be played back, A signal processing device comprising: an audio output unit that outputs the first audio signal or the second audio signal according to the determination result.

[0116] [I-2] The determination unit, If both the first audio signal and the second audio signal are reproducible, it is determined that the second audio signal should be reproducible. The signal processing device according to I-1, which determines that the first audio signal should be played back if the second audio signal cannot be played back.

[0117] [I-3] The signal processing device according to I-1,2, wherein the determination unit makes a determination according to the status of the first network and / or the second network.

[0118] [I-4] The signal processing apparatus according to I-1,3, wherein the determination unit makes a determination according to the reception quality of at least one of the received video signal, first audio signal, and second audio signal.

[0119] [I-5] The aforementioned audio output unit is If it is determined that the first audio signal should be played back, the missing portion of the first audio signal is filled in using the second audio signal, and / or The signal processing device according to I-1 to I-7, which, when it is determined that the second audio signal should be reproduced, uses the first audio signal to fill in any missing parts of the second audio signal.

[0120] [I-6] The signal processing device according to I-5, wherein the audio output unit synchronizes and complements the first audio signal and the second audio signal.

[0121] [I-7] The signal processing apparatus according to I-1 to I-6, comprising a synchronization processing unit for synchronizing the first signal and the second audio signal.

[0122] [J-1] (Eighth embodiment) A first receiving unit that receives the first signal via a broadcast network, A second receiving unit that receives a second signal of the same type as the first signal via a communication network, A determination unit that determines which of the first signal and the second signal should be reproduced, A signal processing device comprising: an output unit that outputs the first signal or the second signal according to the determination result.

[0123] [J-2] The signal processing apparatus according to J-1, wherein the first signal and the second signal are both video signals or both are audio signals.

[0124] [J-3] The output unit is, If it is determined that the first signal should be reproduced, the missing portion of the first signal is filled in using the second signal, and / or The signal processing device according to J-2, wherein, when it is determined that the second signal should be reproduced, the missing portion of the second signal is filled in using the first signal.

[0125] [K-1] (Eighth embodiment) A first receiving unit that receives a first signal via a first network, A second receiving unit that receives a second signal of the same type as the first signal via a second network, A signal processing device comprising: an output unit that outputs the first signal while supplementing it with the second signal.

[0126] [L-1] (Ninth Embodiment) A first receiving unit receives a first signal containing timing information via a first network, A second receiving unit receives a second signal containing timing information, which is to be reproduced in synchronization with the first signal, via a second network. A synchronization unit that synchronizes the first signal and the second signal based on the aforementioned timing information, A signal processing device comprising: an output unit that removes at least a portion of the timing information from the synchronized first signal and the second signal and outputs it;

[0127] [L-2] The output destination of the output unit is the signal processing device described in L-1, which is a recording medium.

[0128] [M-1] (Tenth embodiment) A signal processing device comprising a display control unit that displays the reception strength of a broadcast network and a communication network in two dimensions on a display at a specific point in time.

[0129] [M-2] The signal processing device according to M-1, wherein the display control unit performs a two-dimensional display with the reception strength of the network as the first axis and the reception strength of the communication network as the second axis.

[0130] [M-3] A first receiving unit that receives the first signal via a broadcast network, The system includes a second receiving unit that receives a second signal to be reproduced in synchronization with the first signal via a communication network, The signal processing apparatus according to M-1,2, wherein the display control unit also displays target values ​​on the display for the reception strength of the broadcast network for reproducing the first signal and the reception strength of the communication network for reproducing the second signal.

[0131] [M-4] The signal processing device according to M-1 to M-3, wherein the display control unit displays the reception strength of the broadcast network and the reception strength of the communication network in real time on a display in two dimensions.

[0132] [M-5] The signal processing device according to M-1 to M-4, wherein the display control unit starts a two-dimensional display in response to user operation, and also displays the maximum value of the reception strength of the broadcast network and the communication network from the time of this start onwards on the display.

[0133] In the above, the first and second networks are, for example, broadcasting networks and communication networks (such as the Internet), but may also be other networks capable of transmitting signals, and except in specific cases, they do not matter whether they are one-way or two-way communications. Broadcasting networks include terrestrial digital broadcasting, BS broadcasting, CS broadcasting, cable broadcasting, etc., and there are no restrictions on their type. Furthermore, communication networks may be multicast or unicast, except in specific cases. Signal processing devices include, for example, televisions, recorders, smartphones, mobile phones, tablets, amplifiers, tuners, car navigation systems, and personal computers.

[0134] The first and second signals are, for example, video signals or audio signals, but may also be control signals, except in specific cases. Both the first and second signals may be video signals, for example, video from one viewpoint and video from another viewpoint in 3D video or multi-view video, or a part and another part in high-resolution video. Also, both the first and second signals may be audio signals. Furthermore, each of the first and second signals may contain multiple signals (also video signals, audio signals, etc.) through multiplexing or other means. [Explanation of Symbols]

[0135] 2 Television

Claims

1. A first means for receiving a first video signal, A second means for receiving a second video signal, A signal processing device that is an in-car television, comprising: a third means for reproducing the first video signal while interpolating the missing portion of the first video signal using the second video signal.

2. A first means for receiving a first video signal, A second means for receiving a second video signal, A signal processing device, which is a smartphone or tablet, comprising: a third means for reproducing the first video signal while interpolating the missing portion of the first video signal using the second video signal.

Images