Broadcast reception terminal device
The broadcast receiving terminal device addresses the issue of irrelevant images by controlling the display of inserted content based on the viewer's audio selection, improving the viewing experience by hiding or showing images accordingly.
Patent Information
- Application Number
- JP2024135151
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-13
AI Technical Summary
In television broadcasting, viewers selecting a secondary audio channel are presented with inserted images related to the main audio, which are often irrelevant and can be distracting.
A broadcast receiving terminal device that controls the display of inserted images based on the viewer's selected audio channel, allowing images to be hidden or displayed in conjunction with the audio mode, using a front-end processing unit to generate and control video, audio, and text/graphic data streams, and an audio control unit to manage audio output.
Enables viewers to selectively view or hide inserted images related to the selected audio channel, enhancing the viewing experience by eliminating irrelevant content.
Smart Images

Figure 2024161463000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a broadcast receiving terminal device, and more particularly to a broadcast receiving terminal device adapted to a digital broadcast service system. [Background technology]
[0002] In television broadcasting services, broadcasters use various techniques to enhance the programs they provide in order to increase the interest of viewers. For example, in sports programs, a method is adopted in which a commentator gives commentary while synthesizing images handwritten by a commentator using a so-called touch panel computer with real-time images captured by a television camera on the scene. More specifically, in golf programs, a commentator gives commentary while handwriting-inputting images of the slope of the green, the direction of the grain of the grass, and the target putting line, superimposed on the camera image.
[0003] A known technology for synthesizing such camera images and hand-drawn images is, for example, the following Patent Document 1. That is, Patent Document 1 discloses an image synthesis device that synthesizes a manually input image and an image captured by a camera in conjunction with the camera parameters of the camera.
[0004] On the other hand, in the current television broadcasting service, a program is sometimes provided by audio multiplex broadcasting (e.g., dual audio broadcasting) in which a plurality of audio signals are multiplexed into one program. For example, in audio multiplex broadcasting, the audio of a commentator in a program such as the one described above is generally broadcast using a main audio channel, and another commentator or guest gives a commentary while having a dialogue on a switchable secondary audio channel. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2007-323418 A Summary of the Invention [Problem to be solved by the invention]
[0006] In a program using the above-mentioned technology of combining camera footage and handwritten images input with a pen, even if a viewer selects a secondary audio channel, the image input with a pen by the commentator on the main audio channel is displayed on the television screen, as shown in Fig. 14. Therefore, for a viewer who selects the secondary audio channel under multiplex audio broadcasting, the image input with a pen and inserted into the camera footage is meaningless as it has no relation to the contents of the secondary audio channel, and can sometimes be perceived as a nuisance.
[0007] In view of the above, the present invention aims to provide a broadcasting service that can hide insert images that are not related to the selected audio channel in the display content of a program provided to a viewer.
[0008] More specifically, one of the objects of the present invention is to provide a broadcast receiving terminal device and a display control method which, in a program in which camera footage and other inserted images are superimposed and provided to the viewer, enables the inserted image to be displayed or hidden in conjunction with the audio channel (audio mode) selected by the viewer.
[0009] Another object of the present invention is to provide a broadcast receiving terminal device and a display control method which enable a viewer to selectively display or hide an insert image, even if the insert image is related to an audio channel, in a program in which camera footage is superimposed and combined with other insert images to be provided to the viewer. [Means for solving the problem]
[0010] The present invention, which aims to solve the above problems, includes the following invention-specific matters and technical features.
[0011] The present invention according to one aspect is a broadcast receiving terminal device adapted for a digital broadcast service using a broadcast signal based on a program stream in which a plurality of streams based on program material data constituting a program are multiplexed. The broadcast receiving terminal device includes a front-end processing unit that generates the program material data including at least video data, first audio data, second audio data, and character and graphic data based on the broadcast signal of the received program, a display control unit that controls so that a program image is displayed on a screen of a display device based on display data, and an audio control unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker. The display control unit generates the display data by superimposing the character and graphic data on the video data in conjunction with the output of either the first audio data or the second audio data selected according to a viewer's operation.
[0012] The front-end processing unit may include a demodulation unit that demodulates the broadcast signal into the program stream, a separation unit that separates the demodulated program stream into at least a video stream, a first audio stream, a second audio stream, and a text / graphics stream, and a decoding unit that decodes the video stream, the first audio stream, the second audio stream, and the text / graphics stream.
[0013] The display control unit may control the display so that the character and graphic data is superimposed on the video data when the first audio data is selected, and may control the display so that the character and graphic data is not superimposed on the video data when the second audio data is selected.
[0014] In addition, the display control unit can perform control so that, when a further operation is performed by the viewer after the first audio data is selected, the text and graphic data is not superimposed on the video data.
[0015] The present invention according to another aspect is a broadcast receiving terminal device adapted for a digital broadcast service. The broadcast receiving terminal device includes a front-end processing unit that generates at least video data, first audio data, second audio data, and character and graphic data based on a broadcast signal of a received program, a display control unit that controls so that a program image of either a first form or a second form is displayed on a screen of a display device, and an audio control unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker. Here, the program image of the first form is an image including at least a moving image based on the video data and a character and graphic image based on the character and graphic data. Also, the program image of the second form is an image including the moving image but not the character and graphic image, and the display control unit controls so that the program image of the first form is displayed on the screen when the audio based on the first audio data is controlled to be output from the speaker. Also, the display control unit controls so that the program image of the second form is displayed on the screen when the audio based on the second audio data is controlled to be output from the speaker.
[0016] The display control unit may include a switching control unit that controls whether or not to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. The switching control unit may control the character and graphic data to be output to the synthesis unit when the sound based on the first audio data is controlled to be output from the speaker. The switching control unit may control the character and graphic data not to be output to the synthesis unit when the sound based on the second audio data is controlled to be output from the speaker.
[0017] The character / graphic data may include at least first character / graphic data and second character / graphic data. The switching control unit may control the first character / graphic data to be output to the synthesis unit when the program control information generated based on the broadcast signal includes information designating display of a character / graphic image in a first audio mode. Also, the switching control unit controls the second character / graphic data to be output to the synthesis unit when the program control information includes information designating display of a character / graphic image in a second audio mode.
[0018] Another aspect of the present invention is a broadcast receiving terminal adapted for a digital broadcast service. The broadcast receiving terminal includes a front-end processor that generates at least video data, audio data, and character and graphic data based on a broadcast signal of a program to be received, a display controller that controls so that a program image of either a first form or a second form is displayed on a screen of a display device, and an audio controller that controls so that audio based on the audio data is output from a speaker. Here, the program image of the first form is an image that includes at least a moving image based on the video data and a character and graphic image based on the character and graphic data, and the program image of the second form is an image that includes the moving image but does not include the character and graphic image. When the program is configured in either a monaural broadcasting format or a stereo broadcasting format, the display controller controls so that the program image of either the first form or the second form is selectively displayed on the screen in response to an operation of an audio switching button of a remote controller associated with the broadcast receiving terminal.
[0019] The display control unit may include a switching control unit that controls whether or not to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. The switching control unit may control whether or not to output the character and graphic data to the synthesis unit each time the audio switching button is operated.
[0020] According to another aspect of the present invention, there is provided a broadcast receiving terminal device adapted for a digital broadcast service. The broadcast receiving terminal device includes a front-end processing unit that generates at least video data, first audio data, second audio data, character and graphic data, and program control information based on a broadcast signal of a received program, a display control unit that controls a program image of either a first form or a second form to be displayed on a screen of a display device, and an audio processing unit that controls an audio based on either the first audio data or the second audio data to be output from a speaker according to an audio mode. Here, the program image of the first form is an image including at least a moving image based on the video data and a character and graphic image based on the character and graphic data, and the program image of the second form is an image including the moving image but not including the character and graphic image. When the program control information includes information specifying the display of a character and graphic image in a first audio mode, the display control unit controls the program image of the first form to be displayed on the screen, and the audio processing unit controls the audio based on the first audio data to be output from the speaker. On the other hand, when the program control information includes information specifying the display of a text / graphic image in a second audio mode, the display control unit controls the program image in the first form to be displayed on the screen, and the audio processing unit controls the audio based on the second audio data to be output from the speaker.
[0021] The display control unit may include a switching control unit that controls whether to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. The switching control unit may control the character and graphic data to be output to the synthesis unit when the program control information includes information specifying display of a character and graphic image in the first audio mode. On the other hand, the switching control unit may control the character and graphic data not to be output to the synthesis unit when the program control information includes information specifying display of a character and graphic image in the second audio mode.
[0022] The present invention can be realized as a method invention, and can also be realized as a computer program for causing a computer to execute the method, and as a recording medium having the program recorded thereon.
[0023] In this disclosure, "means" does not simply mean physical means, but also includes cases where the functions of the means are realized by software. Also, the functions of one means may be realized by two or more physical means, and the functions of two or more means may be realized by one physical means.
[0024] In addition, in this disclosure, a "system" includes a logical collection of multiple devices (or functional modules that realize a specific function), regardless of whether each device or functional module is physically configured as a single entity or as separate entities. Effect of the Invention
[0025] According to the present invention, it is possible to hide insert images that are not related to the selected audio channel in the display content of a program provided to a viewer. In particular, in a program in which a camera image and another insert image are superimposed and synthesized to be provided to a viewer, it is possible to display or hide the insert image in conjunction with the audio channel (audio mode) selected by the viewer.
[0026] Furthermore, according to the present invention, in a program in which camera footage and other insert images are superimposed and synthesized to be provided to the viewer, the viewer can selectively display or hide the insert image, even if the insert image is related to an audio channel.
[0027] Other technical features, objects, and operational effects or advantages of the present invention will become apparent from the following embodiments described with reference to the accompanying drawings. The effects described in this disclosure are illustrative and are not limited thereto, and other effects may be present. [Brief description of the drawings]
[0028] [Figure 1] FIG. 1 is a diagram for explaining an example of an overall scheme of a digital broadcasting service system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a schematic configuration of a broadcasting provider system in a digital broadcasting service system according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to a first embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing an example of processing in the broadcast receiving terminal device of the digital broadcast service system according to the first embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram for explaining the transition of setting states caused by the audio switch button in the broadcast receiving terminal device of the digital broadcast service system according to the first embodiment of the present invention. [Figure 6] FIG. 6 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the second embodiment of the present invention. [Figure 7] FIG. 7 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. [Figure 9] FIG. 9 is a diagram for explaining the transition of setting states caused by the audio switch button in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the fourth embodiment of the present invention. [Figure 11]FIG. 11 is a block diagram showing an example of a schematic configuration of a broadcasting company system in a digital broadcasting service system according to a fifth embodiment of the present invention. [Figure 12] FIG. 12 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to a fifth embodiment of the present invention. [Figure 13] FIG. 13 is a flowchart showing an example of processing in a broadcast receiving terminal device in a digital broadcast service system according to a modified example of the fifth embodiment of the present invention. [Figure 14] FIG. 14 is a diagram for explaining an example of a main audio and a sub-audio in a dual audio broadcast with respect to a program image displayed on the screen of a broadcast receiving terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0029] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and application of techniques not specified below. The present invention can be implemented by various modifications (for example, combining the respective embodiments) within the scope of the gist of the present invention. In addition, in the following description of the drawings, the same or similar parts are represented by the same or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. The drawings may also include parts with different dimensional relationships and ratios.
[0030] [First embodiment] This embodiment is characterized in that, in a digital broadcasting service using a broadcasting signal based on a transmission stream in which a plurality of digital data streams constituting program content are multiplexed, when the audio mode of a broadcast receiving terminal device is a first audio mode (e.g., main audio mode), a character and graphic image is controlled to be displayed superimposed on a main moving image (e.g., a video image from a television camera) constituting an image (program image) of the program content on a screen, and when the audio mode is a second audio mode (e.g., secondary audio mode), the character and graphic image is controlled not to be displayed superimposed on the moving image. Note that in this disclosure, a program image in which a character and graphic image is superimposed on a moving image may be referred to as a program image of the first type, and a program image in which a character and graphic image is not superimposed on a moving image may be referred to as a program image of the second type.
[0031] FIG. 1 is a diagram for explaining an example of an overall scheme of a digital broadcasting service system according to an embodiment of the present invention. As shown in the figure, the digital broadcasting service system 1 of this embodiment is configured to include a broadcasting company system 10 and a broadcasting receiving terminal device 20. In the following, the digital broadcasting service system 1 will be described using a terrestrial digital broadcasting service as an example, but is not limited to this, and may be a satellite broadcasting service such as a so-called BS broadcasting service using a broadcasting satellite S or a CS broadcasting service using a communication satellite. Alternatively, the digital broadcasting service system 1 may be a so-called cable television broadcasting service via a distribution company C or an IPTV service using an IP network.
[0032] The broadcasting company system 10 may be configured to include broadcasting facilities for television broadcasting and radio broadcasting. The broadcasting company system 10 is typically managed and operated by a broadcasting company (e.g., a broadcasting station) that has been granted a broadcasting license, but is not limited to this. Although not shown, the broadcasting company system 10 broadcasts program content stored in a transmission server over airwaves under the control of an automatic operation device, for example, and broadcasts (e.g., terrestrial digital broadcasting).
[0033] Program contents are packaged as a "program" that is organized along a time axis and viewed by a viewer, and may be used synonymously with "program" depending on the context. Program contents may include main program contents and secondary data broadcasting program contents. Program contents may be composed of a collection of various program material data (mono-media data) such as moving images (video), subtitles (and superimposed text), text and graphic images, and / or audio. Program material data is converted and transmitted into a multiplexed digital data stream (hereinafter referred to as "stream") conforming to the MPEG2 format, which is a high compression encoding technology, but is not limited to this. An MPEG-2 transport stream (TS) is one form of stream consisting of a series of TS packets. From the viewpoint of transmission technology, a multiplexed stream may be called a transmission stream. In this disclosure, a stream or data related to program contents may be called a "program stream" or "program data". The broadcasting company system 10 generates a program stream by multiplexing each of a plurality of streams based on program material data together with predetermined service information in accordance with program scheduling along a time axis, and further modulates this into a broadcast signal in a predetermined frequency band and transmits it as a broadcast wave from a transmitting base station 12, such as a radio tower, to the target broadcast area.
[0034] The audio broadcasting method of a broadcast program can be classified into monaural broadcasting (including so-called mono-stereo broadcasting) and audio multiplex broadcasting according to the usage of the audio channel, and the audio multiplex broadcasting can be further classified into stereo broadcasting and dual audio broadcasting. Dual audio broadcasting is a multiplex broadcasting consisting of a main audio and a secondary audio. In this embodiment, a program is described on the premise that the audio broadcasting method is dual audio broadcasting.
[0035] The broadcast receiving terminal device 20 is a terminal device for a viewer (user) to view a program provided by a broadcaster. The broadcast receiving terminal device 20 may typically be a so-called television receiver or a set-top box, but is not limited to this. In other words, the broadcast receiving terminal device 20 may be any device that has a function that allows a viewer to view a desired program, and may be, for example, a computing device such as a desktop computer, a notebook computer, a tablet computer, a smartphone, a feature phone, or other intelligent devices.
[0036] 2 is a block diagram showing an example of a schematic configuration of a broadcasting provider system in a digital broadcasting service system according to the first embodiment of the present invention. As shown in the figure, the broadcasting provider system 10 is composed of components such as a program operation device 110, an encoding unit 120, a text and graphic inserting unit 130, a program information generating unit 140, a multiplexing unit 150, a scrambling unit 160, a modulating unit 170, and a transmitting unit 180.
[0037] The program operation device 110 outputs program material data constituting program content stored in a transmission server (not shown) to the encoding unit 120 in accordance with program scheduling. The program operation device 110 also outputs various control information related to the program (hereinafter referred to as "program control information") to the program information generating unit 140. The program control information includes, for example, information on what kind of stream the program content is made up of, what kind of address information is assigned to it, and so on.
[0038] The encoding unit 120 generates a stream by encoding (compressing and encoding) various types of program material data. The encoding unit 120 includes, for example, a video encoding unit 121, a still image encoding unit 122, an audio encoding unit 123, a subtitle encoding unit 124, and a text and graphics encoding unit 125.
[0039] The video encoding unit 121 encodes video (moving image / picture) data related to the program content using a codec conforming to the MPEG-2 video standard, for example, thereby generating a video stream in PES format. The video data related to the program content is displayed on the screen as the main image (moving image), and includes, for example, video data shot by a television camera and video data to which a production element has been added by the producer (broadcast station). The video encoding unit 121 outputs the generated video stream to the multiplexing unit 150.
[0040] In the past, for example, a video stream obtained by superimposing text and graphic images handwritten by a commentator using a tablet computer on a video captured by a television camera on a broadcasting company's side constitutes the display contents (main moving image) of a program. In this disclosure, as described below, such text and graphic images are treated as a stream (text and graphic stream) separate from the video stream based on the video captured by the television camera, as necessary.
[0041] The still image encoding unit 122 encodes still image data related to the data broadcasting program content using a predetermined codec, for example, to generate a still image stream in the PES format. The still image encoding unit 122 converts and encodes still image data in the JPEG format, for example, into video data in units of several seconds (for example, 30 seconds), thereby generating a pseudo still image stream. The still image encoding unit 122 outputs the generated still image stream to the multiplexing unit 150.
[0042] The audio encoding unit 123 encodes audio data related to the program content using a codec conforming to the MPEG-2 audio standard, for example, thereby generating an audio stream in the PES format. The audio stream may include a first audio stream and a second audio stream. A primary audio stream corresponding to the primary audio mode (primary audio channel) is one form of the first audio stream, and a secondary audio stream corresponding to the secondary audio mode (secondary audio channel) is one form of the second audio stream. The audio encoding unit 123 outputs the generated audio stream to the multiplexing unit 150.
[0043] The subtitle encoding unit 124 encodes the subtitle and / or superimposed text data related to the program content in accordance with, for example, a broadcast subtitle file exchange format, thereby generating a stream in the PES format. The subtitle encoding unit 124 outputs the generated subtitle stream to the multiplexing unit 150.
[0044] The character and graphic encoding unit 125 receives character and graphic data input from the character and graphic insertion unit 130 described later, encodes the data, and generates a character and graphic stream in the PES format. The character and graphic encoding unit 125 generates a character and graphic stream by converting, for example, JPEG-format character and graphic data into video data in units of several seconds (for example, 30 seconds). The character and graphic encoding unit 125 outputs the generated character and graphic stream to the multiplexing unit 150.
[0045] The character / graphics insertion unit 130 is an interface for inserting character / graphics data related to program contents into the program contents. The character / graphics data is image data expressing characters and / or graphics, and should not be interpreted in a restrictive manner. The character / graphics insertion unit 130 is configured to include, for example, a touch panel computer operated by a staff member producing a program or a performer of the program. As an example, in a sports program broadcast, the character / graphics insertion unit 130 accepts information input by a commentator using a pen while commentating in the program being broadcast using a touch panel computer. That is, in the present disclosure, a character / graphics image that was previously synthesized with an original video by a broadcasting company and formed a part of the video is treated independently of the video by the viewer as necessary. The character / graphics insertion unit 130 outputs the accepted character / graphics stream to the character / graphics encoding unit 125. Note that in this example, the character / graphics insertion unit 130 is configured as a component separate from the character / graphics encoding unit 125, but for example, the character / graphics insertion unit 130 may be configured to have the function of the character / graphics encoding unit 125.
[0046] The program information generating unit 140 generates a section-format stream based on the program control information. The program control information includes, for example, information about program organization. Program sequence information in MPEG-2 is one form of program control information. The program sequence information includes program specific information (PSI) and service information (SI) that complements the PSI.
[0047] The multiplexing unit 150 generates a multiplexed program stream (transmission stream) based on the various streams described above. A multiplexer is one form of the multiplexing unit 150. As described above, an MPEG-2 transport stream (TS) is one form of a transmission stream made up of a series of TS packets. The multiplexing unit 150 outputs the generated program stream to the scrambler 160.
[0048] The scrambling unit 160 scrambles the generated program stream using an encryption key. Scrambling is mainly performed to realize limited reception for the purpose of protecting the rights of the program content. Such a limited reception method is called CAS (Conditional Access System). The scrambling unit 160 outputs the scrambled program stream to the modulation unit 170.
[0049] The modulation unit 170 digitally modulates the scrambled program stream to generate a broadcast signal in a predetermined frequency band assigned to each broadcaster. The modulation unit 170 outputs the modulated broadcast signal to the transmission unit 180.
[0050] The transmitting unit 180 emits the modulated broadcast signal as a broadcast wave from, for example, a radio tower toward a predetermined broadcast target area in accordance with a predetermined broadcast output. Alternatively, the broadcast wave is emitted toward a broadcast satellite S or the like, and transmitted from the broadcast satellite S to the ground. Alternatively, the transmitting unit 180 may transmit the broadcast signal to, for example, a distributor C that operates a cable television broadcasting service.
[0051] Although not shown, the broadcaster system 10 may include a component for providing data broadcast program content to viewers. Data related to the data broadcast program content may also be multiplexed into the program stream. The data broadcast program content may be linked content linked to the main program content, or may be independent content independent of the main program content.
[0052] 3 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to the first embodiment of the present invention. As shown in the figure, the broadcast receiving terminal device 20 includes, for example, a tuner / modulation unit 210, a descrambling unit 220, a separation unit 230, a decoding unit 240, an output control unit 250, a data broadcast engine 260, a control unit 270, and an operation reception unit 280. Also, as shown in the figure, the broadcast receiving terminal device 20 is connected to a display device DSP and a speaker SPK as output devices.
[0053] The tuner / modulator 210 tunes (selects) the broadcast waves received by an antenna (not shown) to extract a desired broadcast signal, and further demodulates it to obtain a program stream. The program stream at this stage is in a scrambled state. Furthermore, the tuner / modulator 210 performs error correction processing when demodulating the program stream. In this example, the tuner / modulator 210 is configured as one component, but is not limited to this and may be configured as separate components. The tuner / modulator 210 outputs the demodulated program stream to the descramble 220.
[0054] The descrambling unit 220 descrambles (decodes) the demodulated program stream using a decryption key, and restores the original multiplexed program stream. The decryption key is read from, for example, an IC card (not shown) called a CAS card that is attached to the broadcast receiving terminal device. The descrambling unit 220 outputs the descrambled program stream to the separation unit 230.
[0055] The separation unit 230 separates the descrambled multiplexed program stream into each stream. A demultiplexer is one form of the separation unit 230. The separation unit 230 outputs each stream separated from the program stream to the decoding unit 240, the data broadcasting engine 260, and / or the control unit 270 according to the stream. For example, the separation unit 230 outputs the video stream, the audio stream, the subtitle stream, and the text and graphics stream separated from the program stream to the decoding unit 240, while outputting the stream for data broadcasting to the data broadcasting engine 260. The separation unit 230 also outputs the separated program control information to the control unit 270.
[0056] The decoding unit 240 decodes each stream to generate program material data that constitutes the program content. The decoding unit 240 includes, for example, a video decoding unit 241, an audio decoding unit 242, a still image decoding unit 243, a subtitle decoding unit 244, and a character and graphics decoding unit 245.
[0057] The video decoding unit 241 decodes the video stream using a codec conforming to the MPEG-2 video standard, for example, to generate video data. The video decoding unit 241 outputs the decoded video data to the output control unit 250. As will be described later, the video data input to the output control unit 250 is selectively and superimposed with other data and converted into a video signal by a display driver (not shown) and output to the display device DSP. The display device displays the video of the program based on the video signal. The display device DSP is, for example, a liquid crystal display (LCD) or an organic electroluminescence display (OLED), but is not limited to these.
[0058] The still image decoding unit 243 decodes the still image stream to generate still image data. The still image decoding unit 243 outputs the decoded still image data to the output control unit 250. The decoded still image data is displayed on the screen of a display device as, for example, a data broadcast image.
[0059] The audio decoding unit 242 decodes the audio stream according to, for example, the MPEG-2 audio standard, thereby generating audio data. As described above, in the present disclosure, the audio stream may include a first audio stream and a second audio stream. Therefore, the audio decoding unit 242 may generate first audio data corresponding to a main audio mode and second audio data corresponding to a secondary audio mode based on the first audio stream and the second audio stream, respectively. At least one of the decoded first audio data and second audio data is output to an audio driver (not shown) via an audio selection unit 252 (described later), and is further converted by the audio driver into an audio signal and output from the speaker SPK as audio (voice) in the audible range.
[0060] The subtitle decoding unit 244 decodes the subtitle stream according to, for example, a broadcast subtitle file interchange format, thereby generating subtitle data. The subtitle decoding unit 244 outputs the generated subtitle data to the output control unit 250. The subtitle data output to the output control unit 250 is selectively superimposed on the video data and becomes part of the display content of the program.
[0061] The character and graphic decode unit 245 decodes the character and graphic stream to generate character and graphic data. The character and graphic decode unit 245 outputs the generated character and graphic data to the output control unit 250. The character and graphic data output to the output control unit 250 is selectively superimposed on the video data. In this embodiment, the character and graphic data is superimposed on the video data only in the main voice mode.
[0062] In the present disclosure, the tuner / modulation unit 210, the descrambling unit 220, the separation unit 230, and the decoding unit 240 constitute a front-end processing unit. That is, the front-end processing unit is understood as a component that generates various program material data (mono-media data) such as video data and audio data, program control information, and the like, based on a received broadcast signal.
[0063] The output control unit 250 controls the output to the display device DSP and / or the speaker SPK under the control of the control unit 270. In the present disclosure, the output control unit 250 can be considered as a component including a display control unit that controls the output to the display device DSP and an audio control unit that controls the output to the speaker SPK. Under the control of the control unit 270, the output control unit 250 performs control so that either a program image of a first form in which a character and graphic image based on character and graphic data is superimposed on a moving image based on video data, or a program image of a second form in which a character and graphic image is not superimposed on a moving image, is displayed on the screen of the display device DSP.
[0064] That is, the display control section in the output control section 250 generates display data by selectively and superimposing other program material data related to display onto the video data under the control of the control section 270. Furthermore, the display control section converts the display data into a video signal of an appropriate format by a display driver, and drives the display device DSP based on the video signal so that a program image is displayed on the display device DSP.
[0065] Also, the audio control unit in the output control unit 250 selects and outputs at least one of the decoded first audio data and second audio data. Furthermore, the audio control unit converts the output audio data into an audio signal by an audio driver, thereby driving the speaker SPK.
[0066] In the figure, a moving image / still image switching unit 251, an audio selection unit 252, switches 253a and 253b, and a synthesis unit 254 are illustrated as components of the output control unit 250. The moving image / still image switching unit 251, the switches 253a and 253b, and the synthesis unit 254 are examples of a display control unit, and the audio selection unit 252 is an example of an audio control unit.
[0067] The moving image / still image switching unit 251 selects and outputs either video data or still image data. The moving image / still image switching unit 251 selectively switches, for example, under the control of the data broadcasting engine 260. Alternatively, the moving image / still image switching unit 251 may be configured to select both video data and still image data, and output video data in which they are appropriately laid out within one screen.
[0068] The audio selection unit 252 selects and outputs the first audio data and / or the second audio data under the control of the control unit 270. In this embodiment, the audio selection unit 252 selects and outputs either the first audio data or the second audio data. Alternatively, in the case of program content configured as a stereo broadcast, the audio selection unit 252 selects and outputs both the first audio data and the second audio data. As will be described later, in this embodiment, the selection operation of the audio selection unit 252 is controlled so as to be linked with the on / off operation of the switch 253b.
[0069] The switch 253a, under the control of the control unit 270, switches whether to output or not output the subtitle data to the synthesis unit 254. Also, the switch 253b, under the control of the control unit 270, switches whether to output or not output the character and graphic data to the synthesis unit 254. The switches 253a and 253b are one form of a switching control unit.
[0070] The synthesizer 254 synthesizes other program material data related to display with the video data (and / or still image data) in a superimposed manner, thereby generating display data. The data is synthesized by, for example, alpha blending using a transparency information coefficient (alpha value). In this example, the synthesizer 254 includes a synthesizer 254a for synthesizing subtitle data with the video data, and a synthesizer 254b for further synthesizing character and graphic data with the output of the synthesizer. When there is no output of subtitle data and / or character and graphic data, Null data is synthesized with the video data, and the output is the video data itself. The display data output by the synthesizer 254 is converted into a video signal by a display driver (not shown) and output to a display device.
[0071] The data broadcasting engine 260 decodes the separated data broadcasting stream, thereby generating data broadcasting display data and a display control signal. The data broadcasting engine 260 analyzes the display control signal, and controls the moving image / still image switching unit 251 based on the analysis result.
[0072] The operation reception unit 280 receives operation instructions from the viewer to the broadcast receiving terminal device 20. For example, the viewer may give instructions to the broadcast receiving terminal device 20 by operating a remote controller in hand. The viewer's operation instructions include, for example, an instruction to switch channels (broadcast stations), an instruction to control the volume, an instruction to display / hide data broadcast program content, an instruction to display / hide subtitles, and an instruction to switch audio modes. The viewer selectively switches the audio mode between the main audio mode and the secondary audio mode by, for example, pressing an audio switch button. The operation reception unit 280 outputs the received operation instructions to the control unit 270.
[0073] The control unit 270 performs overall control of the operation of the broadcast receiving terminal device 20. Although not shown, the control unit 270 is configured to include a processor module including a CPU and a memory module including various memories. Furthermore, under the control of the CPU, the control unit 270 can control the operation of applications installed in the broadcast receiving terminal device 20. The applications may be installed in advance in the broadcast receiving terminal device 20, or may be downloaded and installed by broadcast waves or via a communication network.
[0074] For example, the control unit 270 determines the audio broadcasting method of the selected program based on the program control information. In this embodiment, the control unit 270 determines whether the audio broadcasting method of the program is dual audio broadcasting based on the program control information. The control unit 270 also stores in memory the state of the current audio mode selected by the audio switch button.
[0075] Furthermore, when the control unit 270 receives an instruction to switch the audio mode via the operation receiving unit 280, it controls the audio selection unit 252 and also controls the switch 253b. More specifically, in this embodiment, when the main audio mode is selected by the instruction to switch the audio mode, the control unit 270 controls the audio selection unit 252 to select the first audio data (the second audio data is not selected), and controls the switch 253b to turn on so that the character and graphic data is output to the synthesis unit 254. On the other hand, when the secondary audio mode is selected by the instruction to switch the audio mode, the control unit 270 controls the audio selection unit 252 to select the second audio data (the first audio data is not selected), and controls the switch 253b to turn off so that the character and graphic data is not output to the synthesis unit 254.
[0076] 4 is a flowchart showing an example of a process in a broadcast receiving terminal device of a digital broadcast service system according to the first embodiment of the present invention. More specifically, the figure shows an example of a process flow until the broadcast receiving terminal device 1 provides a program to a viewer based on a received broadcast wave.
[0077] As shown in the figure, the broadcast receiving terminal device 20 tunes (selects a channel) a broadcast wave received by an antenna (not shown) in the tuner / modulator 210, thereby extracting a desired broadcast signal, and further demodulates it to obtain a scrambled program stream (S401). The scrambled program stream is output to the descramble unit 220.
[0078] Next, the broadcast receiving terminal device 20 descrambles the demodulated program stream using a decryption key in the descrambling section 220, and restores the original multiplexed program stream (S402). Furthermore, the broadcast receiving terminal device 20 separates the descrambled program stream into each stream by the separation section 230 (S403). Each separated stream is output to the decoding section 240. As described above, the separated stream may include at least one of a still image stream, a subtitle stream, and a text and graphic stream, in addition to the video stream, the first audio stream, and the second audio stream. Furthermore, the data broadcast stream of each separated stream is output to the data broadcast engine 260.
[0079] The broadcast receiving terminal device 20 then decodes each of the separated streams in the decoding section 240, thereby generating each of the program material data (S404). As described above, the program material data may include at least one of still image data, subtitle data, and text and graphic data, in addition to the video data, the first audio data, and the second audio data. In this disclosure, the processes from S401 to S404 may be collectively referred to as "front-end processing."
[0080] The broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program in the control unit 270 (S405). That is, the control unit 270 determines whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or is another audio broadcast (monaural broadcast / stereo broadcast) based on the program control information.
[0081] When the broadcast receiving terminal device 20 judges that the audio broadcasting method is dual audio broadcasting, it further judges whether the current audio mode is the main audio mode or the secondary audio mode (S406). When the broadcast receiving terminal device 20 judges that the current audio mode is the main audio mode, it selects and outputs the first audio data and turns on the output of the character and graphic data (S407). On the other hand, when the broadcast receiving terminal device 20 judges that the current audio mode is the secondary audio mode, it selects and outputs the second audio data and turns off the output of the character and graphic data (S408).
[0082] Next, the broadcast receiving terminal device 20 superimposes the subtitle data and / or character and graphic data on the video data in the output control unit 250 to generate display data (S409). If there is no output of subtitle data and / or character and graphic data, Null data is superimposed on the video data.
[0083] The broadcast receiving terminal device 20 controls the display device DSP so that a screen based on the output display data is displayed, and also controls the speaker SPK so that audible sound is output based on the output audio data (S410).
[0084] If the broadcast receiving terminal device 20 determines in S405 that the broadcast is an audio broadcast (monaural / stereo broadcast) other than a dual audio broadcast, it superimposes subtitle data and / or text and graphic data on the video data in accordance with the current setting state, generates display data (S409), and controls display and audio output (S410). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform processing in accordance with the mono / stereo broadcast.
[0085] In the main audio mode, the broadcast receiving terminal device 20 of the present embodiment configured as above selects the first audio data under the control of the control unit 270, and outputs the character and graphic data to the synthesis unit 254 to synthesize the data in a superimposed manner on the video data. As a result, audio based on the first audio data is output from the speaker SPK, while the display device DSP displays a program image in a form (first form) in which a character and graphic image (e.g., an image input by handwriting pen) based on the character and graphic data is superimposed on a moving image (e.g., an image from a television camera) based on the video data. On the other hand, when the viewer operates the audio switching button of the remote controller, for example, to select the secondary audio mode, the broadcast receiving terminal device 20 selects the second audio data under the control of the control unit 270, and turns off the output of the character and graphic data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a program image in a form (second form) in which a character and graphic image is not superimposed on a moving image. Therefore, the viewer can watch the program without the video being disturbed by a character and graphic image unrelated to the audio.
[0086] [Second embodiment] This embodiment is a variation of the first embodiment, and is characterized in that in a digital broadcasting service using broadcast signals based on a multiplexed transmission stream, when the audio mode of the broadcast receiving terminal device is a specific audio mode (e.g., main audio mode), the viewer is able to selectively switch between displaying or not displaying text and graphic images that should be superimposed on the main moving image of a program on the screen (e.g., a television camera image).
[0087] That is, the broadcast receiving terminal device 20 of this embodiment is configured to be able to switch between display / non-display of character and graphic images together with switching of the audio mode, for example, every time the viewer operates the audio switch button of the remote controller. More specifically, the broadcast receiving terminal device 20 performs control so that, every time the audio switch button is pressed, the setting state (hereinafter referred to as "setting state") by the audio switch button is cyclically switched to a state ST1 in which display of character and graphic images is permitted in the main audio mode, a state ST2 in which display of character and graphic images is not permitted in the main audio mode, and a state ST3 in which display of character and graphic images is not permitted in the secondary audio mode, as shown in Fig. 5.
[0088] For this reason, in this embodiment, the control unit 270 stores the previous audio mode and display / non-display state of the character / graphic image, i.e., the state of the audio selection unit 252 and the switch 253b, in, for example, a non-volatile memory (not shown). Such a function of storing and reproducing the previous state is sometimes called a last memory function.
[0089] For example, assume that the current setting state is state ST1. That is, the main sound is output from the speaker, and the character and graphic data superimposed on the moving image is displayed on the display device DSP. In state ST1, when the viewer presses the sound switch button once, the control unit 270 changes the setting state to state ST2. Therefore, in order to hide the character and graphic image while keeping the sound mode in the main sound mode, the control unit 270 controls the switch 253b to be turned off without controlling the audio selection unit 252. As a result, the main sound is output from the speaker, and the moving image without the character and graphic image superimposed is displayed on the display device DSP.
[0090] Next, when the viewer presses the audio switching button once in state ST2, the control unit 270 changes the setting state to state ST3. Therefore, the control unit 270 controls the audio selection unit 252 to switch the audio mode from the main audio mode to the secondary audio mode. At this time, the switch 253b remains off. As a result, the secondary audio is output from the speaker, and a moving image without superimposed text and graphic images is displayed on the display device DSP.
[0091] Next, in state ST3, when the viewer presses the audio switching button once, the control unit 270 returns the setting state to state ST1. Therefore, in order to switch the audio mode from the secondary audio mode to the primary audio mode, the control unit 270 controls the audio selection unit 252 and turns on the switch 253b. As a result, the primary audio is output from the speaker, and a character and graphic image superimposed on the moving image is displayed on the display device DSP.
[0092] FIG. 6 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the second embodiment of the present invention.
[0093] As shown in the figure, the broadcast receiving terminal device 20 that receives the broadcast wave performs front-end processing (S601). That is, as described above, the broadcast receiving terminal device 20 tunes (selects a channel) the received broadcast to extract the desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream, and decodes each separated stream, thereby generating each program material data. In addition, the program control information is output to the control unit 270.
[0094] At this time, the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program in the control unit 270 (S602). That is, the control unit 270 determines whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or another audio broadcast (monaural broadcast / stereo broadcast), based on the program control information.
[0095] If the broadcast receiving terminal device 20 determines that the audio broadcasting method is dual audio broadcasting, it then determines whether the current audio mode is the main audio mode or the secondary audio mode (S603). If the broadcast receiving terminal device 20 determines that the current audio mode is the main audio mode, it then determines the current display state of the character and graphic image (S604).
[0096] When the broadcast receiving terminal device 20 judges that the current display state of the character and graphic image is display, it selects and outputs the first audio data and turns on the output of the character and graphic data (S605). On the other hand, when the broadcast receiving terminal device 20 judges that the current display state of the character and graphic image is not display, it selects and outputs the first audio data and turns off the output of the character and graphic data (S606).
[0097] On the other hand, if the broadcast receiving terminal device 20 determines in S603 that the current audio mode is the sub-audio mode, it selects and outputs the second audio data and turns off the output of character and graphic data (S607).
[0098] Next, the broadcast receiving terminal device 20 superimposes the subtitle data and / or character and graphic data on the video data in the output control unit 250 to generate display data (S608). If there is no output of subtitle data and / or character and graphic data, Null data is superimposed on the video data.
[0099] The broadcast receiving terminal device 20 controls so that a screen based on the output display data is displayed on the display device, and also controls so that audible sound is output from a speaker based on the output audio data (S609).
[0100] If the broadcast receiving terminal device 20 determines in S602 that the broadcast is an audio broadcast (monaural / stereo broadcast) other than a dual audio broadcast, it superimposes subtitle data and / or text and graphic data on the video data in accordance with the current settings, generates display data (S608), and controls display and audio output (S609). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform processing in accordance with the mono / stereo broadcast.
[0101] The broadcast receiving terminal device 20 of the present embodiment configured as described above operates to switch between displaying and hiding the character / graphic image linked to the audio mode every time the audio switch button is operated. Therefore, the viewer can display or hide the character / graphic image even if it is related to the audio mode.
[0102] [Third embodiment] This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using a broadcast signal based on a multiplexed transmission stream, text and graphic images are specified in advance to be superimposed on the main moving image of the program on the screen in either a first audio mode (e.g., main audio mode) or a second audio mode (e.g., secondary audio mode), and the broadcast receiving terminal device accordingly displays the text and graphic images superimposed on the main moving image of the program on the screen.
[0103] To realize the digital broadcasting service of this embodiment, the broadcaster system 10 transmits program control information including a descriptor indicating whether a text and graphic image is to be displayed in a main audio mode or a secondary audio mode, included in the program stream. As an example, a descriptor for specifying whether a text and graphic image is to be displayed in a main audio mode or a secondary audio mode is defined in an EIT (Event Information Table) of program arrangement information. The program information generator 140 of the broadcaster system 10 generates a stream based on the program control information including such a descriptor in the EIT, and outputs it to the multiplexer 150. The multiplexer 150 generates and outputs a program stream by multiplexing each stream.
[0104] The demultiplexer 230 of the broadcast receiving terminal device 20 outputs program control information obtained by demultiplexing such a multiplexed program stream to the control unit 270. The control unit 270 refers to the EIT in the program control information and controls the audio selection unit 252 and the switch 253b according to the descriptor of the EIT.
[0105] 7 and 8 are flowcharts showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. Note that Fig. 8 shows details of the processing S703 shown in Fig. 7.
[0106] As shown in Fig. 7, the broadcast receiving terminal device 20 that has received a broadcast wave performs front-end processing (S701). That is, as described above, the broadcast receiving terminal device 20 tunes (selects a channel) the received broadcast, extracts a desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream, and decodes each separated stream, thereby generating each program material data. In addition, the program control information is output to the control unit 270.
[0107] At this time, the control unit 270 of the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program (S702). That is, the control unit 270 determines whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or another audio broadcast (monaural broadcast / stereo broadcast), based on the program control information.
[0108] When the broadcast receiving terminal device 20 judges that the audio broadcasting system is dual audio broadcasting, the control unit 270 then performs output control processing of character and graphic data based on the audio mode and program control information (S703).
[0109] That is, in the output control process of character and graphic data, the broadcast receiving terminal device 20 judges which is the current audio mode as shown in Fig. 8 (S801). If the broadcast receiving terminal device 20 judges that the current audio mode is the main audio mode, it then judges whether or not the descriptor of the program control information specifies that there is display in the main audio mode (S802). If the broadcast receiving terminal device 20 judges that the descriptor of the program control information specifies that there is display in the main audio mode, it selects and outputs the first audio data and turns on the output of the character and graphic data (S803). On the other hand, if the broadcast receiving terminal device 20 judges that the descriptor of the program control information specifies that there is no display in the main audio mode (i.e., specifies that there is display in the secondary audio mode), it selects and outputs the first audio data and turns off the output of the character and graphic data (S804).
[0110] On the other hand, if the broadcast receiving terminal device 20 determines in S801 that the current audio mode is not the primary audio mode (it is the secondary audio mode), it then determines whether or not the descriptor in the program control information specifies that there is display in the secondary audio mode (S805). If the broadcast receiving terminal device 20 determines that the descriptor in the program control information specifies that there is display in the secondary audio mode, it selects and outputs the second audio data and turns on the output of the character and graphic data (S806). On the other hand, if the broadcast receiving terminal device 20 determines that the descriptor in the program control information specifies that there is no display in the secondary audio mode (i.e., specifies that there is display in the primary audio mode), it selects and outputs the second audio data and turns off the output of the character and graphic data (S807).
[0111] Returning to Fig. 7, the broadcast receiving terminal device 20 superimposes the subtitle data and / or character and graphic data on the video data to generate display data (S704). If there is no output of subtitle data and / or character and graphic data, Null data is superimposed on the video data. Next, the broadcast receiving terminal device 20 controls the display device so that a screen based on the output display data is displayed, and controls the speaker so that audible sound is output based on the output audio data (S705).
[0112] If the broadcast receiving terminal device 20 determines in S702 that the broadcast is an audio broadcast (monaural broadcast / stereo broadcast) other than a dual audio broadcast, it superimposes subtitle data and / or text and graphic data on the video data in accordance with the current settings, generates display data (S704), and controls display and audio output (S705). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform control in accordance with a dual audio broadcast.
[0113] As described above, according to this embodiment, the character and graphic image is displayed on the screen of the broadcast receiving terminal device 20 only in the audio mode designated by the broadcasting company. That is, when the character and graphic data is designated to be displayed in the main audio mode, the broadcast receiving terminal device 20 selects the first audio data and outputs the character and graphic data to the synthesis unit 254 to synthesize it in a superimposed manner on the video data only when the current audio mode is the main audio mode. As a result, audio based on the first audio data is output from the speaker SPK, and the display device DSP displays a screen in which, for example, a character and graphic image input by handwriting pen is superimposed on an image captured by a television camera. On the other hand, when the secondary audio mode is selected, the broadcast receiving terminal device 20 selects the second audio data and turns off the output of the character and graphic data. As a result, the speaker SPK outputs audio based on the second audio data, while the display device DSP displays a screen of an image captured by a television camera, and the character and graphic image is no longer displayed.
[0114] Also, when the character and graphic data is designated to be displayed in the secondary audio mode, the broadcast receiving terminal device 20 selects the second audio data and outputs the character and graphic data to the synthesis unit 254 to synthesize it in a superimposed manner on the video data only when the current audio mode is the secondary audio mode. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a screen in which, for example, a character and graphic image input by handwriting pen is superimposed on an image captured by a television camera. On the other hand, when the broadcast receiving terminal device 20 is in the main audio mode, it selects the second audio data and turns off the output of the character and graphic data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a screen of an image captured by a television camera, and the character and graphic image is no longer displayed.
[0115] (Modification) As a variation of this embodiment, as shown in the second embodiment, the broadcast receiving terminal device 20 may be controlled so that the character and graphic images superimposed on the main video image of the program on the screen can be selectively displayed or hidden in response to an operation by the viewer.
[0116] That is, the broadcast receiving terminal device 20 of this embodiment is configured to switch between display / non-display of character and graphic images together with switching of the audio mode, for example, every time the viewer operates the audio switch button of the remote controller. More specifically, the broadcast receiving terminal device 20 performs control so that every time the audio switch button is operated, the state is cyclically switched to a state ST31 in which display of character and graphic images is permitted in the main audio mode, a state ST32 in which display of character and graphic images is not permitted in the main audio mode, a state ST33 in which display of character and graphic images is permitted in the secondary audio mode, and a state ST34 in which display of character and graphic images is not permitted in the secondary audio mode, as shown in Fig. 9.
[0117] For this reason, the control unit 270 realizes a last memory function by storing the previous audio mode and the display / non-display state of the text and graphic image, i.e., the state of the audio selection unit 252 and the switch 253b, for example in a non-volatile memory (not shown).
[0118] For example, suppose that the program control information descriptor of the selected program specifies that text and graphic images are displayed in the secondary audio mode. Also, suppose that the current setting state is state ST31. Therefore, the main audio is output from the speaker SPK, and a moving image without superimposed text and graphic images is displayed on the display device DSP. In state ST31, when the viewer presses the audio switch button once, the control unit 270 changes the setting state to state ST32. That is, since the program control information descriptor does not permit display of text and graphic images in the main audio mode, the control unit 270 transitions the setting state by one, but does not control the audio selection unit 252 and the switch 253b. Therefore, there is no change in the output of the speaker SPK and the display of text and graphic images on the display device DSP.
[0119] Next, when the viewer presses the audio switching button once in state ST32, the control unit 270 changes the setting state to state ST33. That is, since the display of character and graphic images in the secondary audio mode is permitted by the descriptor of the program control information, the control unit 270 controls the audio selection unit 252 to switch to the secondary audio output and turns on the switch 253b in order to switch the audio mode from the main audio mode to the secondary audio mode. As a result, the secondary audio is output from the speaker, and the display device DSP displays the moving image on which the character and graphic images are superimposed.
[0120] Next, when the viewer presses the audio switch button once in state ST33, the control unit 270 changes the setting state to state ST34. Therefore, although the display of character and graphic images in the secondary audio mode is permitted by the descriptor of the program control information, the control unit 270 controls the switch 253b to be OFF in response to the viewer's selection not to display the character and graphic data. As a result, the secondary audio is output from the speaker SPK, and the display device DSP displays a moving image without superimposed character and graphic images.
[0121] Next, in state ST34, when the viewer presses the audio switching button once, the control unit 270 returns the setting state to state ST31. Therefore, the control unit 270 controls the audio selection unit 252 to switch to the secondary audio output in order to switch the audio mode from the primary audio mode to the secondary audio mode. As a result, the primary audio is output from the speaker SPK, and a moving image without a character and graphic image superimposed is displayed on the display device DSP.
[0122] As described above, the broadcast receiving terminal device 20 operates to switch between displaying and hiding the character / graphic image linked to the audio mode every time the audio switch button is operated. Therefore, the viewer can display or hide the character / graphic image even if it is related to the audio mode.
[0123] [Fourth embodiment] This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using broadcast signals based on a multiplexed transmission stream, when a program is configured in either a mono or stereo audio broadcasting format, control is performed so that a character or graphic image is superimposed on the main moving image of the program on the screen (e.g., a television camera image) in response to the operation of the audio switch button on the broadcast receiving terminal device, or control is performed so that the character or graphic image is not displayed on the screen.
[0124] That is, in the conventional broadcast receiving terminal device, the audio switching button on the remote controller is disabled (does not respond to operations) for programs in the monaural or stereo broadcasting format. Therefore, the broadcast receiving terminal device 20 of the present embodiment is configured to switch between displaying and not displaying character and graphic images using the audio mode switching button on the remote controller for programs in the monaural or stereo broadcasting format.
[0125] 10 is a flow chart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to a fourth embodiment of the present invention. This processing is executed when the program selected by the broadcast receiving terminal device 20 is either a monaural broadcasting system or a stereo broadcasting system. Since the processing by the broadcast receiving terminal device 20 is the same regardless of whether the program is a monaural broadcasting system or a stereo broadcasting system, the following description will be given assuming that the program is a monaural broadcasting system. Also, the setting state of the audio switch button of the broadcast receiving terminal device 20 is either a state in which a character and graphic image is displayed or a state in which it is not displayed.
[0126] As shown in the figure, the broadcast receiving terminal device 20 that receives the broadcast wave performs front-end processing (S1001). That is, as described above, the broadcast receiving terminal device 20 tunes (selects) the received broadcast to extract the desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream and decodes each separated stream, thereby generating each program material data.
[0127] At this time, the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program in the control unit 270 (S1002). That is, the control unit 270 determines whether the audio broadcasting format of the selected program is monaural broadcasting / stereo broadcasting or dual audio broadcasting based on the program control information.
[0128] If the broadcast receiving terminal device 20 determines that the audio broadcasting format is monaural / stereo broadcasting, it then determines whether the current setting is display or non-display (S1003). If the broadcast receiving terminal device 20 determines that the current setting is display, it turns on the output of character and graphic data (S1004). On the other hand, if the broadcast receiving terminal device 20 determines that the current setting is non-display, it turns off the output of character and graphic data (S1005). In other words, in this embodiment, the broadcast receiving terminal device 20 does not switch the audio data to be output.
[0129] Next, the broadcast receiving terminal device 20 superimposes the subtitle data and / or character and graphic data on the video data to generate display data (S1006). If there is no output of subtitle data and / or character and graphic data, Null data is superimposed on the video data.
[0130] The broadcast receiving terminal device 20 controls so that a screen based on the output display data is displayed on the display device, and also controls so that audible sound is output from a speaker based on the output audio data (S1007).
[0131] If the broadcast receiving terminal device 20 determines in S1002 that the audio broadcasting format is not monaural / stereo broadcasting, it superimposes subtitle data and / or text and graphic data onto the video data in accordance with the current settings, generates display data (S1006), and controls display and audio output (S1007). As described in other embodiments, the broadcast receiving terminal device 20 may perform processing in accordance with monaural / stereo broadcasting.
[0132] As described above, according to this embodiment, when a program is in either a mono broadcast format or a stereo broadcast format, the viewer can use the audio mode switching button provided on the broadcast receiving terminal device 20 and / or the remote controller to superimpose text and graphic images on the main moving image of the program on the screen, or hide the text and graphic images.
[0133] [Fifth embodiment] This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using a broadcast signal based on a multiplexed transmission stream, a stream based on character and graphic images corresponding to each of a plurality of audio modes (e.g., main audio mode and secondary audio mode) is included in the transmission stream and broadcast, and the character and graphic images corresponding to the selected audio mode are displayed superimposed on the main moving image on the screen of the broadcast receiving terminal device.
[0134] FIG. 11 is a block diagram showing an example of a schematic configuration of a broadcasting company system in a digital broadcasting service system according to a fifth embodiment of the present invention.
[0135] As shown in the figure, the broadcaster system 10' of this embodiment differs from that shown in the above embodiment in that a character / graphics inserting unit 130' outputs first character / graphics data and second character / graphics data corresponding to the main audio and secondary audio, respectively, to a character / graphics encoding unit 125', which in turn outputs a first character / graphics stream and a second character / graphics stream, respectively. A multiplexing unit 150 multiplexes the first character / graphics stream and the second character / graphics stream together with other streams. The other components are the same as those in the above embodiment, and therefore will not be described.
[0136] FIG. 12 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to a fifth embodiment of the present invention.
[0137] As shown in the figure, the broadcast receiving terminal device 20' of this embodiment differs from the above-mentioned embodiment in that a character / graphics decoding unit 245 decodes each of the first character / graphics stream and the second character / graphics stream separated from the program stream by the separation unit 230, thereby outputting the first character / graphics data and the second character / graphics data, and in that a character / graphics selection unit 255 is provided instead of the switch 253b.
[0138] That is, under the control of the control unit 270, the character / graphic selection unit 255 outputs either the first character / graphic data or the second character / graphic data to the synthesis unit 254 in conjunction with the selection operation of the audio selection unit 252. That is, when the viewer selects the main audio mode, under the control of the control unit 270, the audio selection unit 252 selects the first audio data, and the character / graphic selection unit 255 selects the corresponding first character / graphic data. On the other hand, when the viewer selects the secondary audio mode, under the control of the control unit 270, the audio selection unit 252 selects the second audio data, and the character / graphic selection unit 255 selects the corresponding second character / graphic data.
[0139] As described above, according to this embodiment, the broadcasting company transmits text and graphic data corresponding to each audio mode in the transmission stream, so that the text and graphic images corresponding to the audio mode selected by the viewer are appropriately superimposed on the main moving image of the program on the screen.
[0140] That is, the character and graphic image is displayed on the screen of the broadcast receiving terminal device 20 only in the audio mode designated by the broadcasting company. More specifically, when the current audio mode is the main audio mode, the broadcast receiving terminal device 20 selects the first audio data and also selects the corresponding first character and graphic data, outputs it to the synthesis unit 254, and synthesizes it in a superimposed manner on the video data. As a result, audio based on the first audio data is output from the speaker SPK, while the display device DSP displays a screen in which, for example, the first character and graphic image corresponding to the audio mode is superimposed on the video captured by the television camera.
[0141] Furthermore, when the current audio mode is the secondary audio mode, the broadcast receiving terminal device 20 selects the second audio data and also selects the second character and graphic data, outputs it to the synthesis unit 254, and synthesizes it in a superimposed manner on the video data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a screen in which, for example, a second character and graphic image input by handwriting pen is superimposed on the video captured by the television camera.
[0142] (Modification) As a variation of this embodiment, the broadcast receiving terminal device 20 may be controlled so that, in response to operation by the viewer, the audio mode can be switched and text and graphic images superimposed on the main moving image of the program on the screen can be selectively displayed or hidden.
[0143] That is, the broadcast receiving terminal device 20 of this embodiment is configured to switch between display / non-display of either the first character / graphic image or the second character / graphic image corresponding to the audio mode, together with switching the audio mode, each time the audio switching button of the remote controller is operated by the viewer. In this case, as shown in the modified example of the third embodiment (FIG. 9), the broadcast receiving terminal device 20 performs control to cyclically switch between states ST31 to ST34 each time the audio switching button is operated.
[0144] For example, assume that the current setting state is state ST31. Therefore, the main sound is output from the speaker SPK, and the moving image on which the first character and graphic image is superimposed is displayed on the display device DSP. In state ST31, when the viewer presses the sound switch button once, the control unit 270 changes the setting state to state ST32. Therefore, the control unit 270 controls the character and graphic selection unit 255 to be turned off. As a result, the output from the speaker SPK remains the main sound, and the moving image on which the first character and graphic image is not superimposed is displayed on the display device DSP.
[0145] Next, when the viewer presses the audio switching button once in state ST32, the control unit 270 changes the setting state to state ST33. Therefore, in order to switch the audio mode from the main audio mode to the secondary audio mode, the control unit 270 controls the audio selection unit 252 to switch to output the secondary audio, and controls the character / graphic selection unit 255 to select the second character / graphic image. As a result, the secondary audio is output from the speaker SPK, and the display device DSP displays the moving image on which the second character / graphic image is superimposed.
[0146] Next, when the viewer presses the audio switch button once in state ST33, the control unit 270 changes the setting state to state ST34. Therefore, the control unit 270 controls the character / graphic selection unit 255 to be turned off. As a result, the output from the speaker SPK remains the secondary audio, and the display device DSP displays a moving image without the second character / graphic image superimposed thereon.
[0147] Next, in state ST34, when the viewer presses the audio switching button once, the control unit 270 returns the setting state to state ST31. Therefore, in order to switch the audio mode from the main audio mode to the secondary audio mode, the control unit 270 controls the audio selection unit 252 to switch to output the secondary audio, and controls the character / graphic selection unit 255 to select the first character / graphic image. As a result, the main audio is output from the speaker SPK, and the display device DSP displays a moving image on which the first character / graphic image is superimposed.
[0148] Next, the process in the broadcast receiving terminal device of this embodiment will be described. The process in the broadcast receiving terminal device is basically the same as the process shown in Fig. 7, but the output control process of character and graphic data in S703 is different. Fig. 13 is a flow chart showing an example of the process in the broadcast receiving terminal device of the digital broadcast service system according to the fifth embodiment of the present invention, and specifically shows the details of the output control process of character and graphic data corresponding to S703 shown in Fig. 7.
[0149] That is, in the output control process of the character / graphic data of this embodiment, as shown in Fig. 13, the broadcast receiving terminal device 20 judges which is the current audio mode (S1301). When the broadcast receiving terminal device 20 judges that the current audio mode is the main audio mode, it then judges whether or not the setting state of the audio switch button specifies that the first character / graphic image is displayed (S1302). When the broadcast receiving terminal device 20 judges that the setting state specifies that the first character / graphic image is displayed, it selects and outputs the first audio data and turns on the output of the first character / graphic data (S1303). On the other hand, when the broadcast receiving terminal device 20 judges that the setting state specifies that the first character / graphic image is not displayed, it selects and outputs the first audio data and turns off the output of the first character / graphic data and the second character / graphic data (S1304).
[0150] On the other hand, in S1301, when the broadcast receiving terminal device 20 judges that the current audio mode is not the main audio mode (it is the secondary audio mode), it then judges whether or not the setting state of the audio switch button specifies that the second character / graphic image is displayed (S1305). When the broadcast receiving terminal device 20 judges that the setting state specifies that the second character / graphic image is displayed, it selects and outputs the second audio data and also selects and outputs the second character / graphic data (S1306). On the other hand, when the broadcast receiving terminal device 20 judges that the setting state specifies that the second character / graphic image is not displayed (No in S1305), it selects and outputs the second audio data and turns off the output of the first character / graphic data and the second character / graphic data (S1307).
[0151] As described above, the broadcast receiving terminal device 20 operates to switch between displaying and hiding the character / graphic image linked to the audio mode every time the audio switch button is operated. Therefore, the viewer can display or hide the character / graphic image even if it is associated with the audio mode.
[0152] The above-described embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. The present invention can be embodied in various forms without departing from the gist of the present invention.
[0153] For example, in the methods disclosed herein, steps, operations, or functions may be performed in parallel or in a different order, provided that the results are consistent. The steps, operations, and functions described are provided merely as examples, and some of the steps, operations, and functions may be omitted or combined into one, or other steps, operations, or functions may be added, without departing from the spirit of the invention.
[0154] In addition, although various embodiments are disclosed in this specification, specific features (technical matters) in one embodiment can be added to or replaced with specific features in other embodiments, with appropriate modifications, and such forms are also included in the spirit of the present invention. [Explanation of symbols]
[0155] 1. Digital broadcasting service system 10. Broadcasting System 110…Program operation device 120…Encoder section 121...Video encoding section 122...Still image encoding section 123…Audio encoding section 124…Subtitle encoding section 130…Character and graphic insertion section 140...Program information generation unit 150...Multiplexing section 160…Scramble Section 170…Modulation section 180...Transmitter 20...Broadcast receiving terminal equipment 210…Tuner / modulation section 220…Descrambling section 230...Separation section 240…Decoder 250...Output control section 251…Video / still image switching section 252…Audio selection section 253a, 253b…Switch 254…Composition section 255…Character and shape selection section 260…Data broadcasting engine 270...Control unit 280…Operation reception section DSP…display device SPK…Speaker
Claims
1. A broadcast receiving terminal device adapted for digital broadcasting services, a front-end processing unit that generates at least video data, still image data, first audio data, second audio data, subtitle data, and text and graphic data based on a broadcast signal of a received program; a display control unit that controls the display of a program image on a screen of a display device based on the display data; an audio control unit that performs control so that audio based on either the first audio data or the second audio data is output from a speaker; the display control unit generates the display data by synthesizing the character and graphic data on the video data in a superimposed manner in conjunction with output of either the first audio data or the second audio data selected in accordance with an operation of a viewer. Broadcast receiving terminal equipment.
2. The display control unit is when the first audio data is selected, control is performed so that the character and graphic data is superimposed on the video data; when the second audio data is selected, control is performed so that the character and graphic data is not superimposed on the video data; 2. A broadcast receiving terminal device according to claim 1.
3. the display control unit selects and outputs the video data and / or the still image data.
3. A broadcast receiving terminal device according to claim 1 or 2.
4. A data broadcasting engine is further provided for decoding a data broadcasting stream separated from the broadcasting signal and generating display data and a display control signal for data broadcasting, the data broadcasting engine controls the display control unit to output the video data and / or the still image data based on the display control signal.
4. A broadcast receiving terminal device according to claim 3.
Citation Information
Patent Citations
Image composition device and image composition program
JP2007323418A
