Image processing device and image processing system
The image processing device simplifies the detection and handling of pull-down information by associating it with each frame and transmitting it efficiently, addressing the challenges of processing 24Hz film material as 60Hz image information and improving the handling of mixed content frame rates.
Patent Information
- Application Number
- PCT/JP2024/016771
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-09
- Filing Date
- 2024-05-01
- Publication Date
- 2024-11-14
AI Technical Summary
Conventional image processing systems face difficulties in detecting and handling pull-down information for 24Hz film material when transmitting it as 60Hz image information, leading to complications and delays in processing, especially due to variations in pulldown types and the need for frame correlation detection.
An image processing device equipped with an information mapping unit that associates pulldown information, including a pulldown ratio and identifier, with each frame of image information, allowing easy recognition and synchronization, and inserts this information into the blanking period as IF or EMP data for efficient transmission and processing.
Enables straightforward acquisition and utilization of pull-down information, reducing processing complexity and enabling accurate display of image information without requiring frame correlation detection, thus improving the handling of mixed film and video material frame rates.
Smart Images

Figure JP2024016771_14112024_PF_FP_ABST
Abstract
Description
Image processing device and image processing system
[0001] The present technology relates to an image processing device and an image processing system, and more particularly to an image processing device or the like that handles image information that has been subjected to pull-down processing.
[0002] It has been known to transmit image information from a source device such as a media player to a sink device such as a television (TV) via a High-Definition Multimedia Interface (HDMI) transmission path. The image information in this case includes not only image information that has not been pulled down but also image information that has been pulled down. Note that "HDMI" is a registered trademark.
[0003] For example, image information from 24 Hz film material (movie material) is pulled down and transmitted as 60 Hz image information. This allows the frame rates of both types of image information to be matched when transmitting a mixture of film material and video material, making it possible to transmit without changing the transmission mode and suppressing display disturbances on the receiving side that may occur due to a change in transmission mode.
[0004] Conventionally, as described above, with regard to 60 Hz image information obtained by pulling down image information from 24 Hz film material, it is known that the receiving side detects the pull-down and processes the image information differently from that for image information from 60 Hz video material that has not been pulled down, thereby obtaining image information for display that is appropriate as video material.
[0005] For example, with regard to 60 Hz image information from video material, processing such as frame interpolation is performed using image information from all frames to obtain image information for display, while with regard to 60 Hz image information obtained by pulling down 24 Hz image information from film material, processing such as obtaining image information from the 24 Hz frame before pull-down and then frame multiplexing this 24 Hz frame image information to obtain image information for display is performed.
[0006] Conventionally, when it comes to 60 Hz image information obtained by pull-down of 24 Hz film image information, the receiving side has used frame correlation to detect whether pull-down has been performed, but this detection process takes time and is complicated due to the variations in pull-down. For example, variations in pull-down from 24 Hz to 60 Hz include (a) 2-3 pull-down, (b) 3-2 pull-down, and (c) a mixture of a and b.
[0007] For example, Patent Document 1 discloses a technique for deriving a pulldown period and the number of motion frames from motion detection within a moving image, determining the type of pulldown, and calculating the original frame frequency before pulldown.
[0008] Japanese Patent Application Laid-Open No. 2012-231303
[0009] An object of the present technology is to make it possible to easily obtain and use pull-down information related to image information.
[0010] The concept of this technology is an image processing device that includes an information association unit that associates pull-down information with image information for each frame.
[0011] In this technology, an information associating unit associates pull-down information with image information for each frame. For example, the pull-down information may include a pull-down ratio and an identifier within a repeat sequence. By including the pull-down ratio in the pull-down information in this manner, if the image information has been pulled down, the pull-down ratio can be easily recognized, and a synchronization signal corresponding to the frame rate of the image information before pull-down can be easily generated. Furthermore, by including an identifier within a repeat sequence in the pull-down information in this manner, if the image information has been pulled down, the pull-down period can be easily recognized, and image information of the frame before pull-down can be easily extracted.
[0012] In this case, for example, if the image information is not pulled down, the pull-down ratio may be set to 1:1, which makes it possible to recognize that the image information is not pulled down, since the pull-down ratio is 1:1.
[0013] In this case, the identifier may be formed by combining, for example, a source frame count corresponding to the frame before pulldown and a base frame count corresponding to the frame after pulldown. This makes it easy to recognize that, when image information has been pulled down, a frame where both the source frame count and the base frame count are 1 is the first frame in a pulldown cycle, and also makes it easy to extract image information of a frame before pulldown by extracting image information of a frame where the source frame count changes.
[0014] Here, for example, if the image information is not pulled down, the source frame count may be fixed to 0. This makes it possible to recognize that the image information is not pulled down because the source frame count is 0. This also makes it possible to set the pull-down ratio to one corresponding to the pulled-down image information to be switched from this image information, thereby making it possible to prepare in advance for the pulled-down image information.
[0015] Also, for example, the pull-down information may include flag information indicating whether or not the image information overlaps with the immediately preceding frame. This makes it possible, when the image information has been pulled down, to easily detect whether or not it has been pulled down, without using frame correlation. Also, for example, the pull-down information may include flag information indicating whether or not it has been pulled down. This makes it possible, when the image information has been pulled down, to easily detect that it has been pulled down.
[0016] Alternatively, for example, the information associating unit may insert the pull-down information into a blanking interval of the image information for each frame. In this case, for example, the information associating unit may insert the pull-down information into the blanking interval as IF (InfoFrame) information. In this case, for example, the information associating unit may insert the pull-down information into the blanking interval as EMP (Extended Metadata Packet) information. This makes it easy to associate the pull-down information with the image information for each frame.
[0017] Furthermore, for example, the image display device may further include a transmitter that transmits the image information, in which pull-down information is associated with each frame, to an external device via a transmission path. In this case, the transmission path may be, for example, an HDMI transmission path. This allows the image information, in which pull-down information is associated with each frame, to be transmitted to the external device, and the external device can simply and appropriately process the image information based on the pull-down information to obtain image information for display.
[0018] In this way, in the present technology, pull-down information is associated with image information for each frame, and it is possible to easily obtain and use pull-down information related to image information.
[0019] Another concept of the present technology is an image processing device including a processing unit that obtains second image information of a frame before pull-down from first image information based on pull-down information associated with the first image information for each frame, and obtains image information for display.
[0020] In this technology, a processing unit obtains second image information of a frame before pulldown from first image information based on pulldown information associated with the first image information for each frame, thereby obtaining image information for display.
[0021] For example, the pull-down information may include a pull-down ratio and an identifier in the repeat sequence. In this case, for example, the processing unit may acquire second image information of a frame before pull-down based on the identifier in the repeat sequence, set a frequency division ratio of the frequency divider based on the pull-down ratio to acquire a synchronization signal corresponding to the frame rate of the frame before pull-down, and process the acquired second image information of the frame before pull-down based on the acquired synchronization signal to acquire image information for display. This makes it possible to obtain image information for display in a good condition based on the pull-down ratio included in the pull-down information and the identifier in the repeat sequence.
[0022] Here, for example, the identifier may be configured to include a combination of a source frame count corresponding to the frame before pulldown and a base frame count corresponding to the frame after pulldown, and the processing unit may acquire second image information of the frame before pulldown by extracting image information of a frame corresponding to a change in the source frame count from the first image information, thereby making it possible to appropriately acquire the second image information of the frame before pulldown.
[0023] Also, here, for example, the identifier may be configured to be a combination of a source frame count corresponding to a frame before pull-down and a base frame count corresponding to a frame after pull-down, and the processing unit may synchronize the frequency divider unit at the timing of a frame where both the source frame count and the base frame count are 1. This makes it possible to align the phase of the synchronization signal acquired by the frequency divider unit with the frame phase of the second image information of the frame before pull-down, thereby enabling good synchronization with, for example, audio.
[0024] In this way, in this technology, second image information of the frame before pull-down is obtained from first image information based on pull-down information that is associated with this first image information for each frame, thereby obtaining image information for display. This makes it possible to easily and appropriately obtain image information for display without the need to detect whether the first image information has been pulled down using frame correlation.
[0025] In the present technology, for example, the processing unit may determine whether the first image information has been pulled down based on the pull-down information, and if the first image information has been pulled down, obtain second image information of the frame before the pull-down from the first image information based on the pull-down information to obtain image information for display. This makes it possible to obtain second image information of the frame before the pull-down based on the pull-down information to obtain image information for display only if the first image information has been pulled down.
[0026] In this case, for example, the pulldown information may include a pulldown ratio, and the processing unit may determine that the first image information has not been pulled down if the pulldown ratio is 1:1. Also in this case, for example, the pulldown information may include an identifier in the repeating sequence, and this identifier is configured by a combination of a count of source frames corresponding to frames before pulldown and a count of base frames corresponding to frames after pulldown, and the processing unit may determine that the first image information has not been pulled down if the source frame count is 0.
[0027] Another concept of the present technology resides in an image processing system having a first image processing device and a second image processing device, wherein the first image processing device comprises an information associating unit that associates pull-down information with first image information for each frame, and the second image processing device comprises a processing unit that acquires second image information of a frame before pull-down from the first image information based on the pull-down information associated with the first image information for each frame, to obtain image information for display.
[0028] FIG. 1 is a block diagram showing an example of the configuration of an image processing system according to an embodiment. FIG. 1 is a diagram showing an example of the configuration of a data transmitting unit (HDMI transmitting unit) of a media player and a data receiving unit (HDMI receiving unit) of a television receiver in the image processing system. FIG. 2 is a diagram showing an example of the structure of TMDS transmission data. FIG. 3 is a block diagram showing an example of the configuration of a media player. FIG. 4 is a diagram showing an example of the type of pulldown from 24 Hz to 60 Hz. FIG. 5 is a diagram showing an example of the data structure of an AVI infoframe packet. FIG. 6 is a diagram showing an example of the data structure of a pulldown infoframe packet. FIG. 7 is a diagram showing an example of the pulldown ratio N:M included in a pulldown infoframe associated with each frame in pulldown from 24 Hz to 60 Hz, and an example of counts s and b constituting an identifier in a repeat sequence. FIG. 8 is a diagram for explaining an EMP, which is a packet having a pulldown EM. FIG. 9 is a diagram showing an example of pulldown information associated with each frame of image data with a frame rate of 60 Hz, which is image information of video material that has not been pulled down. FIG. 10 is a diagram showing another example of pulldown information associated with each frame of image data with a frame rate of 60 Hz, which is image information of video material that has not been pulled down. FIG. 1 is a block diagram showing an example of the configuration of a television receiver. FIG. 2 is a block diagram showing an example of the configuration of an image conversion unit and a display control unit. FIG. 3 is a diagram for explaining video processing and film (movie) processing. FIG. 4 is a flowchart showing an example of frame-by-frame control processing for the N / M frequency divider unit and frame extraction unit of the control unit. FIG. 5 is a flowchart showing another example of frame-by-frame control processing for the N / M frequency divider unit and frame extraction unit of the control unit. FIG. 6 is a flowchart showing another example of frame-by-frame control processing for the N / M frequency divider unit and frame extraction unit of the control unit. FIG. 7 is a diagram showing roughly the processing in a television receiver when image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material, is sent from a media player.1 is a block diagram showing another example of the configuration of a television receiver; 2 is a block diagram showing an example of the hardware configuration of a computer; 3 is a diagram showing an example of pulldown; 4 is a diagram showing an example of types of pulldown from 48 Hz to 60 Hz; 5 is a diagram showing an example of a pulldown ratio N:M associated with each frame and counts s and b constituting an identifier in a repeat sequence in pulldown with a plurality of pulldown ratios; 6 is a diagram showing an example of pulldown information associated with each frame in a case where image data (image information) with a frame rate of 24 Hz is pulled down from 24 Hz to 60 Hz (first pulldown) and image data (image information) with a frame rate of 48 Hz is pulled down from 48 Hz to 60 Hz (second pulldown); 7 is a diagram showing an example of pulldown information associated with each frame in a case where image data (image information) with a frame rate of 24 Hz is pulled down from 24 Hz to 48 Hz and image data (image information) with a frame rate of 48 Hz is not pulled down; FIG. 1 is a diagram showing an example of a case where pull-down information associated with image data (image information) for each frame is a duplicate flag constituting flag information indicating whether or not the frame is duplicated with the immediately preceding frame; FIG. 2 is a diagram showing an example of a case where pull-down information associated with image data (image information) for each frame is a pull-down flag constituting flag information indicating whether or not pull-down is performed; FIG. 3 is a diagram showing an example of a case where pull-down information associated with image data (image information) for each frame is both a duplicate flag and a pull-down flag; and FIG. 4 is a diagram showing an example of the data structure of a packet of a Video Frame Structure Infoframe (Video Frame Structure IF) including both a duplicate flag and a pull-down flag.FIG. 10 is a diagram for explaining an EMP (Extended Metadata Packet), which is a packet having a video frame structure EM (Video Frame Structure Extended Metadata) including a duplicate flag and a pull-down flag.
[0029] Modes for carrying out the invention (hereinafter referred to as "embodiments") will be described below. The description will be made in the following order. 1. Embodiments 1-1. Example of the configuration of a transmission / reception system 1-2. Example of the configuration of an HDMI transmission unit / reception unit 1-3. Example of the structure of TMDS transmission data 1-4. Example of the configuration of a media player 1-4-1. AVI infoframe 1-4-2. Correspondence of pull-down information to image data 1-4-3. Support / non-support of pull-down infoframes by sink devices 1-4-4. EMP, a packet having a pull-down EM 1-5. Example of the configuration of a television receiver 1-5-1. Example of the configuration of an image conversion unit and a display control unit 1-5-2. Display operation for received image data in a television receiver 1-5-3. Control processing procedure for an N / M frequency division unit and a frame extraction unit 1-6. Other example of the configuration of a television receiver 1-7. Processing by software 2. Modifications 2-1. Other examples of pulldown 2-2. In the case of 24Hz / 48Hz mixed content 2-3. Other examples of pulldown information 2-4. Other
[0030] 1 shows an example of the configuration of an image processing system 10 according to an embodiment. The image processing system 10 includes a media player 100 as a source device and a television receiver 200 as a sink device, which are connected by an HDMI cable 300. Here, the HDMI cable 300 forms a transmission path.
[0031] The media player 100 associates pull-down information with the image information for each frame, and then transmits the image information, with the pull-down information associated with each frame, to the television receiver 200 via the HDMI cable 300.
[0032] In this embodiment, the pulldown information includes a pulldown ratio and a repeat sequence identifier. The repeat sequence identifier is configured by combining a source frame count corresponding to a frame before pulldown and a base frame count corresponding to a frame after pulldown. The repeat sequence identifier is not limited to the combination of these two counts, and may be configured in any way as long as it can identify each frame of the repeat sequence.
[0033] Even if the image information is not pull-down information, it is possible to associate pull-down information with the image information for each frame, and in this case, the pull-down information is set to, for example, a state that makes it possible to identify that the image information is not pull-down information. For example, the pull-down ratio included in the pull-down information is set to 1:1.
[0034] Also, for example, the count of source frames constituting the identifier of the repeat sequence included in the pulldown information is fixed to 0. By fixing the count of source frames to 0 in this way and making it possible to identify that the image information is not pulled down, it becomes possible to set the pulldown ratio not to 1:1 but to one that corresponds to the pulled-down image information to be switched from this image information, and it becomes possible to prepare in advance for the pulled-down image information.
[0035] In this embodiment, pull-down information is inserted into the blanking interval of image information, thereby associating pull-down information with image information for each frame. For example, the pull-down information is inserted into the blanking interval as IF (InfoFrame) information. Furthermore, in the case of HDMI 2.1, for example, the pull-down information can also be inserted into the blanking interval as EMP (Extended Metadata Packet) information. Note that the method for associating pull-down information with image information for each frame is not limited to the method of inserting pull-down information into the blanking interval of image information, and other methods may also be used.
[0036] When the image information (first image information) received from the media player 100 via the HDMI cable 300 has been pulled down, the television receiver 200 performs a process to obtain image information for display from this image information (first image information) by acquiring image information (second image information) of the frame before the pull-down based on the pull-down information associated with this image information (first image information) for each frame, and then displays the image.
[0037] In this embodiment, the pull-down information includes a pull-down ratio and a repeat sequence identifier, as described above. The television receiver 200 acquires image information of the frame before pull-down (second image information) based on the identifier in the repeat sequence, sets the division ratio of the divider based on the pull-down ratio to acquire a synchronization signal corresponding to the frame rate of the frame before pull-down, and processes the acquired image information of the frame before pull-down (second image information) based on the acquired synchronization signal to obtain image information for display. In this way, by using the pull-down ratio and the repeat sequence identifier included in the pull-down information, it is possible to obtain image information for display in a good condition.
[0038] Furthermore, in this embodiment, the identifier of the repeat sequence is formed by a combination of the source frame count corresponding to the frame before pull-down and the base frame count corresponding to the frame after pull-down, as described above. The television receiver 200 appropriately acquires the image information of the frame before pull-down (second image information) by extracting the image information of the frame corresponding to the change in the source frame count from the image information (first image information). Furthermore, the television receiver 200 synchronizes the frequency divider unit at the timing of the frame where both the source frame count and the base frame count are 1, thereby aligning the phase of the synchronization signal acquired by the frequency divider unit with the frame phase of the image information before pull-down (second image information), thereby enabling good synchronization with, for example, audio.
[0039] Furthermore, the television receiver 200 determines whether the image information received from the media player 100 via the HDMI cable 300 has been pulled down, and if it has not been pulled down, performs processing to obtain image information for display using this image information, and displays the image.
[0040] Here, if there is no pull-down information associated with each frame of the received image information, the television receiver 200 determines that the received image information has not been pulled down. Furthermore, if there is pull-down information associated with each frame of the received image information but the pull-down ratio is 1:1, the television receiver 200 also determines that the received image information has not been pulled down. Furthermore, if there is pull-down information associated with each frame of the received image information but the source frame count is 0, the television receiver 200 also determines that the received image information has not been pulled down.
[0041] 1-2. Example of the Configuration of the HDMI Transmitter / Receiver FIG. 2 shows an example of the configuration of the data transmitter (HDMI transmitter) of the media player 100 and the data receiver (HDMI receiver) of the television receiver 200 in the image processing system 10. As shown in FIG.
[0042] The media player 100 transmits differential signals corresponding to one screen's worth of image data (Vido) over multiple channels to the television receiver 200 during an active image interval (hereinafter also referred to as an "active video interval" where appropriate), which is the interval from one vertical synchronization signal to the next, excluding the horizontal and vertical blanking intervals, and also transmits differential signals corresponding to audio data (Audio) and auxiliary data (Auxiliary) accompanying the image data over multiple channels to the television receiver 200 during horizontal blanking intervals or vertical blanking intervals. The auxiliary data includes an active audio stream, an active video stream, or an InfoFrame for describing the source, and other data.
[0043] That is, the media player 100 has an HDMI transmitter 101. The HDMI transmitter 101 converts, for example, image data into a corresponding differential signal and serially transmits the signal to the television receiver 200 connected via an HDMI cable 300 over three TMDS channels #0, #1, and #2, which are multiple channels.
[0044] In addition, the HDMI transmitter 101 converts audio data and auxiliary data accompanying the image data into corresponding differential signals and transmits them serially to the television receiver 200 connected via the HDMI cable 300 on three TMDS channels #0, #1, and #2.
[0045] Furthermore, the HDMI transmitter 101 transmits, on the TMDS clock channel, a pixel clock synchronized with the image data transmitted on the three TMDS channels #0, #1, and #2 to the television receiver 200 connected via the HDMI cable 300. On one TMDS channel #i (i=0, 1, 2), 10-bit image data is transmitted during one pixel clock cycle.
[0046] TMDS coding is an 8-bit / 10-bit conversion coding that converts 8-bit data into 10-bit data, and by reducing the number of transition points compared with the previous data, it is possible to suppress adverse effects such as unwanted radiation while maintaining DC balance. Therefore, since the coding run length cannot be theoretically guaranteed, DC coupling and separate clock transmission are essential.
[0047] The television receiver 200 receives differential signals corresponding to image data transmitted from the media player 100 over multiple channels during the active video period, and also receives differential signals corresponding to audio data and auxiliary data transmitted from the television receiver 200 over multiple channels during the horizontal blanking period or vertical blanking period.
[0048] That is, the television receiver 200 has an HDMI receiver 201. The HDMI receiver 201 receives differential signals corresponding to image data and differential signals corresponding to audio data and auxiliary data transmitted from the media player 100 connected via an HDMI cable 300 on TMDS channels #0, #1, and #2 in synchronization with a pixel clock transmitted from the media player 100 on the TMDS clock channel.
[0049] In the above example, image data, audio data, and auxiliary data are transmitted over TMDS channels #0, #1, and #2, and the pixel clock is transmitted over the TMDS clock channel. This is compatible with HDMI 1.4 and earlier, as well as HDMI 2.0. In the case of HDMI 2.1, transmission is performed using FRL lanes #0, #1, #2, and #3. In this case, the TMDS clock channel is FRL lane #3.
[0050] In this case, data is transmitted by a fixed rate link (FRL) packet using three lanes #0 to #2 or four lanes #0 to #3. Here, FRL character coding is a 16-bit / 18-bit conversion coding that converts 16-bit data into 18-bit data, maintains DC balance, and allows clock extraction.
[0051] In addition to the above-mentioned TMDS and FRL transmission channels, the HDMI system also includes transmission channels called DDC (Display Data Channel), CEC Line, Utility Line, and HPD Line.
[0052] The DDC is made up of two lines (signal lines) not shown that are included in the HDMI cable 300. The DDC is used by the media player 100 to read EDID stored in an EDID ROM (Extended Display Identification ROM) of the television receiver 200 via the HDMI cable 300. The DDC is also used by the media player 100 to read and write SCDCCS data stored in an SCDC (Status and Control Data Channel) register of the television receiver 200 via the HDMI cable 300.
[0053] The CEC line is used for two-way communication of control data between the media player 100 and the television receiver 200. The HPD line is used by the media player 100 to detect connection of the television receiver 200. The utility line and the HPD line are also used to configure HEAC (HDMI Ethernet and Audio Return Channel) and eARC (Enhanced Audio Return Channel).
[0054] "1-3. Example of TMDS Transmission Data Structure" Figure 3 shows an example of the structure of TMDS transmission data. This example structure shows various transmission data sections when image data of horizontal x vertical B pixels x A lines is transmitted on TMDS channels #0 to #2. In the video field in which transmission data is transmitted on an HDMI TMDS channel, there are three types of sections depending on the type of transmission data. These three types of sections are the video data period, the data island period, and the control period.
[0055] Here, the video field interval is the interval from the rising edge of one vertical synchronization signal to the rising edge of the next vertical synchronization signal. This video field interval is divided into a horizontal blanking period, a vertical blanking period, and an active video period. This active video period is the period obtained by excluding the horizontal and vertical blanking periods from the video field interval, and corresponds to the above-mentioned video data period. During this video data period, data of B pixels by A lines of active pixels that constitute one screen's worth of image data is transmitted.
[0056] The data island section is allocated to a part of the horizontal blanking period and a part of the vertical blanking period. In this data island section, audio data and auxiliary data are packetized and transmitted. The control section is allocated to the other part of the horizontal blanking period and the vertical blanking period. In this control section, vertical synchronization signals, horizontal synchronization signals, etc. are transmitted.
[0057] 1-4. Example of the configuration of a media player Figure 4 shows an example of the configuration of a media player 100. In Figure 4, parts corresponding to those in Figure 2 are designated by the same reference numerals. In addition to an HDMI transmitter 101, the media player 100 has a control unit 102, a pull-down conversion unit 103, and changeover switches 104 and 105.
[0058] The control unit 102 controls the operation of each unit of the media player 100. The pulldown conversion unit 103 pulls down image data Va having a frame rate of 24 Hz, which is image information of film material (movie material), to obtain image data Vb having a frame rate of 60 Hz. There are several types of pulldown from 24 Hz to 60 Hz, and the pulldown conversion unit 103 performs one of these types of pulldown.
[0059] 5(a) to 5(c) show examples of pulldown from 24 Hz to 60 Hz. FIG. 5(a) shows so-called 2-3 pulldown, in which every two frames before pulldown are converted into five frames after pulldown, with the first frame converted into two frames and the second frame converted into three frames. In this figure, frames after pulldown marked with an "*" indicate that they are the same frame as the previous frame (duplicate frame). This is also true for the other figures.
[0060] Figure 5(b) shows a so-called 3-2 pulldown where every two frames before pulldown are converted into five frames after pulldown, with the first frame converted into three frames and the second frame converted into two frames. Figure 5(c) shows a combination of the 2-3 pulldown shown in Figure 5(a) and the 3-2 pulldown shown in Figure 5(b).
[0061] 4, the changeover switch 104 switches between image data Va having a frame rate of 24 Hz that has not been pulled down converted and image data Vb having a frame rate of 60 Hz that has been pulled down converted. In this case, the image data Va is input to the fixed terminal on the a side, the image data Vb is input to the fixed terminal on the b side, and the image data Va or image data Vb is output from the movable terminal as image data Vc.
[0062] The changeover switch 105 switches between image data Vd having a frame rate of 60 Hz, which is image information of video material, and image data Vc output from the changeover switch 104. In this case, the image data Vd is input to the fixed terminal on the a side, and the image data Vc is input to the fixed terminal on the b side, and the image data Vd or the image data Vc is output from the movable terminal as image data Ve and supplied to the HDMI transmitter 101 as image data (Video).
[0063] Here, depending on the switching state of the changeover switches 104 and 105, the image data Ve becomes image data with a frame rate of 60 Hz, which is image information of video material, image data with a frame rate of 24 Hz, which is image information of film material (movie material), or image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material).
[0064] In addition to image data (Video), a horizontal synchronization signal (Hsync) and a vertical synchronization signal (Vsync) corresponding to the image data (Video), as well as audio data (Audio) and auxiliary data (Auxiliary) accompanying the image data (Video) are supplied to the HDMI transmitter 101. The HDMI transmitter 101 combines these data into TMDS transmission data and transmits it to the television receiver 200 via the HDMI cable 300.
[0065] The auxiliary data (Auxiliary) is supplied from the control unit 102 to the HDMI transmitter 101. As described above, this auxiliary data (Auxiliary) includes an active audio stream, an active video stream, or an InfoFrame for explaining the source, and other data.
[0066] "1-4-1. AVI InfoFrame" An InfoFrame included in auxiliary data (Auxiliary) is the AVI InfoFrame (AVI-IF: Auxiliary Video Information InfoFrame). This AVI InfoFrame can transmit supplementary information about an image from a source device to a sink device.
[0067] Figures 6(a) and 6(b) show an example of the data structure of an AVI infoframe packet. This AVI infoframe packet is defined in CTA-861-H, with some specification changes made on the HDMI side. Figure 6(a) shows the packet header (AVI IF Packet Header), and Figure 6(b) shows the packet contents (AVI IF Packet Contents).
[0068] A video format identification code (VIC) is placed in byte PB4 of the packet content. For example, as shown in the example below, the value of the VIC indicates the frame size and frame rate of the transmitted image data (Video).
[0069] VIC=16: 2K60P (1920×1080 / 60P) VIC=32: 2K24P (1920×1080 / 24P) VIC=111: 2K48P (1920×1080 / 48P) VIC=97: 4K60P (3840×2160 / 60P) VIC=93: 4K24P (3840×2160 / 24P) VIC=114: 4K48P (3840×2160 / 48P)
[0070] Although detailed description is omitted, the packet contents also include video-related information such as color space in addition to the above-mentioned video format identification code.
[0071] "1-4-2. Associating pull-down information with image data" In this embodiment, at least when the image data (Video) supplied to the HDMI transmitter 101 is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), pull-down information is associated with this image data (image information) for each frame.
[0072] For this reason, the control unit 102 includes a newly defined pull-down infoframe (Pull-down IF) having pull-down information in the auxiliary data (Auxiliary) supplied to the HDMI transmitter 101. This pull-down infoframe includes, as pull-down information, the pull-down ratio and an identifier within the repeat sequence. Here, the control unit 102 constitutes an information association unit.
[0073] 7(a) and 7(b) show examples of the data structure of a pull-down infoframe packet, where Fig. 7(a) shows the packet header (Pull-down IF Packet Header) and Fig. 7(b) shows the packet contents (Pull-down IF Packet Contents).
[0074] First, the packet header (Pull-down IF Packet Header) shown in FIG. 7(a) will be described. Byte HB0 defines "Packet Type," which indicates the type of packet. Here, this indicates a pull-down infoframe packet. Byte HB1 describes the version of the packet data definition. Byte HB2 describes information indicating the packet length. Here, this is set to "0x04."
[0075] Next, the packet contents (Pull-down IF Packet Contents) shown in Figure 7(b) will be described. The pull-down ratio is described in bytes PB1 and PB2. Byte PB1, "Source Frame Rate Ratio = N," indicates the frame rate ratio N of the source frame, which is the frame before pull-down, and byte PB2, "Base Frame Rate Ratio = M," indicates the frame rate ratio M of the base frame, which is the frame after pull-down. For example, in the case of pull-down from 24 Hz to 60 Hz, the pull-down ratio is 2:5, where N = 2 and M = 5.
[0076] Bytes PB1 and PB2 describe an identifier within the repeating sequence. Here, the identifier is composed of a combination of a source frame count corresponding to the frame before pulldown and a base frame count corresponding to the frame after pulldown. Byte PB3, "Source Frame Count = s," indicates the source frame count corresponding to the frame before pulldown, and byte PB4, "Base Frame Count = b," indicates the base frame count corresponding to the frame after pulldown.
[0077] 8 shows an example of the pulldown ratio N:M included in the pulldown infoframe associated with each frame and the counts s and b constituting the identifier in the repeat sequence in pulldown from 24 Hz to 60 Hz. Note that this example shows the case of 2:3 pulldown.
[0078] In this case, the ratio of the frame rate of the source frame (24 Hz) to the frame rate of the base frame (60 Hz) is 2:5, i.e., the pulldown ratio is 2:5, so N = 2 and M = 5. The repeat sequence is a five-frame cycle from frame 1 to frame 5, with counts s and b being "1, 1" for frame 1, "1, 2" for frame 2, "2, 3" for frame 3, "2, 4" for frame 4, and "2, 5" for frame 5. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 2 (N), and frames with the same count s value are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 5 (M). When s = b = 1, the frame phases before and after pulldown are the same.
[0079] "1-4-3. Pulldown Infoframe Support / Non-Support of Sink Device" Note that with regard to this newly defined pulldown infoframe, a sink device that does not support it will ignore it and continue processing as before, so there will be no compatibility issues. The media player 100 may include the pulldown infoframe in the auxiliary data (Auxiliary) only when the sink device supports the pulldown infoframe.
[0080] For example, the media player 100 may determine whether the sink device supports pull-down infoframes based on flag information that is newly defined on the EDID (Extended Display Identification Data) of the sink device and indicates whether the sink device supports pull-down infoframes.
[0081] 9A shows an example of the structure of the HDMI Forum Vendor Specific Data Block (HF-VSDB), which is newly defined in the Extended Display Identification Data (EDID) in HDMI 2.0. FIG. 9B shows an example of the structure of the Sink Capability Data Structure (SCDS), which is located from the fourth byte onward in the HF-VSDB. For example, a PD-IF flag indicating whether or not pulldown infoframes are supported is newly defined in bit 0 of byte PB11, which is currently reserved. In this case, "0" indicates "not supported" and "1" indicates "supported."
[0082] In this embodiment, the television receiver 200 will be described below assuming that it supports pull-down infoframes.
[0083] "1-4-4. EMP, a packet having pull-down EM" In the above, it has been explained that a newly defined pull-down infoframe having pull-down information is included in the auxiliary data (Auxiliary). However, instead of including a pull-down infoframe in the auxiliary data (Auxiliary), it is also possible to include a pull-down EM (Pull-down Extended Metadata) having pull-down information in the auxiliary data (Auxiliary).
[0084] FIG. 10(a) shows the header (EMP Header) of an EMP (Extended Metadata Packet), which is a packet having a pull-down EM, and FIG. 10(b) shows the contents (EMP Contents) of the EMP.
[0085] The pulldown EM is transmitted in a single packet because it is small in size. Therefore, the fields of the header (EMP Header) are set to the following values: Last=1, First=1, Sequence_Index=0
[0086] Also, since pull-down EM is video frame synchronized in relation to the video format, each field of the contents (EMP Contents) is set to the following values: Sync=1 VFR=1 AFR=0 DS_Type=2: Unique EM Data Set End=1, New=1: Transmitted in a single packet Organization_ID=1: Pull-down EM is an EM specified by HDMI Data_Set_Tag =xxxx: Identification number for Pull-down EM assigned by HDMI Data_Set_Length =4: Pull-down EM uses 4 bytes from MD0...MD3
[0087] 10(c) shows MD (MetaData) content, i.e., pulldown EM, located in bytes PB7-PB27 of the content (EMP Contents). Although detailed description is omitted, bytes MD0-MD3 describe the pulldown ratio (N:M) and the repeat sequence identifier (s, b), similar to bytes PB1-PB4 in the pulldown infoframe packet content shown in FIG. 7(b) above.
[0088] In the above description, the pull-down information is associated with each frame of image data (image information) with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material). However, it is also possible to associate pull-down information with each frame of image data with a frame rate of 60 Hz, which is image information of video material that has not been pulled down, in a state where switching transmission with image data (image information) with a frame rate of 60 Hz obtained by pull-down is performed, i.e., by including a pull-down infoframe in auxiliary data, the sink device can easily recognize that the image data (image information) is with a frame rate of 60 Hz and has not been pulled down.
[0089] 11(a) and (b) show examples of pull-down information associated with 60 Hz frame rate image data, which is image information of video material that has not been pulled down, for each frame. Fig. 11(a) shows a state of switching from 60 Hz frame rate image data obtained by pulling down 24 Hz frame rate image data, which is image information of film material (movie material), to 60 Hz frame rate image data, which is image information of video material that has not been pulled down, and Fig. 11(b) shows a state of switching in the reverse direction.
[0090] The pull-down information associated with each frame of image data with a frame rate of 60 Hz, which is image information of video material that has not been pulled down, will be described below. In this case, the ratio between the frame rate of the source frame and the frame rate of the base frame is 1:1, i.e., the pull-down ratio is 1:1, so N = 1 and M = 1. This allows the television receiver 200, which is the sink device, to determine that the received image data (image information) has not been pulled down based on the fact that N = M = 1. In addition, in this case, the counts s and b that constitute the identifier in the repeat sequence are both fixed to 1.
[0091] 12(a) and 12(b) show another example of pull-down information associated with image data of a frame rate of 60 Hz, which is image information of video material that has not been pulled down, for each frame. Fig. 12(a) shows a state of switching from image data of a frame rate of 60 Hz obtained by pulling down image data of a frame rate of 24 Hz, which is image information of film material (movie material), to image data of a frame rate of 60 Hz, which is image information of video material that has not been pulled down, and Fig. 12(b) shows a state of switching in the reverse direction.
[0092] The following describes pull-down information that is associated with each frame of image data with a frame rate of 60 Hz, which is image information of video material that has not been pulled down. In this case, the count s is fixed to 0. This allows the television receiver 200, which is the sink device, to determine that the received image data (image information) has not been pulled down, based on the fact that the count s = 0.
[0093] In this case, the ratio between the frame rate of the source frame and the frame rate of the base frame is 1:1, i.e., the pulldown ratio is 1:1, but the pulldown ratio N:M indicates a pulldown ratio of 2:5 in the image data with a frame rate of 60 Hz obtained by pulling down the image data with a frame rate of 24 Hz, which is the image information of the film material (movie material), and N=2, M=5.
[0094] In this way, the pulldown ratio N:M indicates a pulldown ratio of 2:5 in image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material). Therefore, it is possible to make preparations for the pulled-down image data in advance, such as preparing a vertical synchronization signal required for image conversion processing and display control processing.
[0095] 1-5. Example of the configuration of a television receiver Figure 13 shows an example of the configuration of a television receiver 200. In Figure 13, parts corresponding to those in Figure 2 are designated by the same reference numerals. In addition to an HDMI receiver 201, the television receiver 200 has a control unit 202, an N / M frequency dividing unit 203, a selector switch 204, a frame extraction unit 205, a selector switch 206, an image conversion unit 207, a display control unit 208, and a display panel 209.
[0096] The control unit 202 controls the operation of each unit of the television receiver 200. The HDMI receiver 201 obtains, in addition to image data (Video), a vertical synchronization signal (Vsync) corresponding to this image data (Video), and audio data (Audio) and auxiliary data (Auxiliary) accompanying this image data (Video) from the TMDS transmission data received from the media player 100. Although not shown in the figure, audio output processing is performed based on the audio data (Audio) obtained by the HDMI receiver 201.
[0097] The auxiliary data (Auxiliary) obtained by the HDMI receiver 201 is supplied to the control unit 202. As described above, the auxiliary data (Auxiliary) includes an AVI infoframe, and at least, when the image data (Video) is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), the auxiliary data includes a pull-down infoframe (Pull-down IF) having pull-down information associated with the image data for each frame.
[0098] Therefore, the control unit 202 acquires a video format identification code from the AVI infoframe and recognizes the picture frame and frame rate of the image data (Video). The control unit 202 also recognizes video-related information, such as the color space of the image data (Video), from the AVI infoframe. The control unit 202 also recognizes from the pulldown infoframe that the image data (Video) is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material), and further recognizes the pulldown ratio N:M and the counts s and b that constitute the identifier within the repeat sequence.
[0099] The vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is supplied to an N / M frequency dividing unit 203. The control unit 202 sets N and M in the N / M frequency dividing unit 203 based on the pulldown ratio N:M recognized from the pulldown infoframe. The N / M frequency dividing unit 203 then divides the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201, that is, the vertical synchronization signal (Vsync) at the frame rate of the image data after pulldown, to obtain a vertical synchronization signal (Vsync') at the frame rate of the image data before pulldown.
[0100] In this case, if the image data (Video) is image data (image information) with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is 60 Hz, but the vertical synchronization signal (Vsync') obtained by the N / M frequency divider unit 203 is 24 Hz.
[0101] The changeover switch 204 switches between the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 and the vertical synchronization signal (Vsync') obtained by the N / M frequency divider 203. In this case, the vertical synchronization signal (Vsync) is input to the fixed terminal on the a side, the vertical synchronization signal (Vsync') is input to the fixed terminal on the b side, and the vertical synchronization signal (Vsync) is output from the movable terminal when the image data (Video) is not pulled down, and the vertical synchronization signal (Vsync') is output when the image data (Video) is pulled down.
[0102] The image data (Video) obtained by the HDMI receiver 201 is also supplied to a frame extraction unit 205. The frame extraction unit 205 extracts, from the image data (Video), image data of frames corresponding to changes in the count s that constitute the identifier within the repeat sequence. The image data of each frame extracted in this manner corresponds to the image data of each frame before pull-down. For example, if the image data (Video) is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material), the frame extraction unit 205 obtains image data with a frame rate of 24 Hz that corresponds to the image data of each frame before pull-down.
[0103] The changeover switch 206 switches between the image data (Video) obtained by the HDMI receiver 201 and the image data extracted by the frame extraction unit 205. In this case, the image data (Video) obtained by the HDMI receiver 201 is input to the fixed terminal on the a side, the image data extracted by the frame extraction unit 205 is input to the fixed terminal on the b side, and from the movable terminal, if the image data (Video) is not pulled down, the image data (Video) obtained by the HDMI receiver 201 is output, and if the image data (Video) is pulled down, the image data extracted by the frame extraction unit 205 is output.
[0104] The image conversion unit 207 processes the image data output from the selector switch 206 in synchronization with the vertical synchronization signal output from the selector switch 204, and does not depend on the display panel 209, etc. The display control unit 208 processes the image data after processing by the image conversion unit 207 in synchronization with the vertical synchronization signal output from the selector switch 204, and does not depend on the display panel 209, etc. The display panel 209 is formed of, for example, a liquid crystal display panel or an organic EL panel, and displays an image based on the image data after processing by the display control unit 208 (image data for display).
[0105] Here, the frame extraction unit 205, image conversion unit 207, display control unit 208, etc. constitute a processing unit that obtains image information for display by obtaining image information of the frame before pull-down based on the pull-down information from the received image information.
[0106] 1-5-1. Example Configurations of the Image Converter and Display Controller Fig. 14 shows an example configuration of the image converter 207 and display controller 208. The image converter 207 has an RGB converter 207a and an EOTF (Electro-Optical Transfer Function) processor 207b. The RGB converter 207a and the EOTF processor 207b perform RGB conversion processing and electro-optical conversion processing based on information relating to input image data, which is included in the AVI infoframe and indicates whether the data is RGB data or YCbCr data, color space information, quantization range information, and the like.
[0107] The display control unit 208 has a display frame conversion unit 208 a, a display frequency conversion unit 208 b, an image quality improvement processing unit 208 c, and a display RGB conversion unit 208 d. The display frame conversion unit 208 a performs display frame conversion processing based on input image data and image frame information of the display panel 209.
[0108] The display frequency conversion unit 208b performs processing to match the frame rate of the input image data with the display frequency of the display panel 209. The image quality improvement processing unit 208c performs image quality improvement processing such as noise reduction on the input image data. The display RGB conversion unit 208d performs color gamut conversion processing on the input image data.
[0109] Here, the processing in the display frequency conversion unit 208b, the image quality improvement processing unit 208c, and the display RGB conversion unit 208d differs depending on whether the input image data is image data with a frame rate of 60 Hz related to video material or image data with a frame rate of 24 Hz related to film material (movie material). That is, if the input image data is image data with a frame rate of 60 Hz related to video material, video processing is performed, while if the input image data is image data with a frame rate of 24 Hz related to film material (movie material), film (movie) processing is performed.
[0110] 15A shows the main differences between video processing and film (movie) processing. Video processing targets image data with a frame rate of 60 Hz, frame interpolation is enabled, the color temperature (white point) is set to 9300 K in color gamut conversion, and noise cancellation is turned on. On the other hand, film (movie) processing targets image data with a frame rate of 24 Hz, frame interpolation is disabled, the color temperature (white point) is set to, for example, 6500 K in color gamut conversion, and noise cancellation is turned off.
[0111] 15B shows an example of video processing in the display frequency conversion unit 208 b, the image quality improvement processing unit 208 c, and the display RGB conversion unit 208 d. The display frequency conversion unit 208 b performs frame interpolation processing on image data of a frame rate of 60 Hz related to the input video material, thereby obtaining image data of a frame rate that matches the display frequency (e.g., 120 Hz) of the display panel 209.
[0112] Furthermore, the image quality improvement processing unit 208c performs image quality improvement processing such as noise reduction on the image data output from the display frequency conversion unit 208b. Furthermore, the display RGB conversion unit 208d performs color gamut conversion processing on the image data output from the image quality improvement processing unit 208c to set the color temperature (white point) to 9300K, thereby obtaining image data for display.
[0113] 15C shows an example of film (movie) processing in the display frequency conversion unit 208 b, image quality improvement processing unit 208 c, and display RGB conversion unit 208 d. The display frequency conversion unit 208 b performs frame multiplexing processing on image data of a frame rate of 24 Hz relating to the input film material (movie material), thereby obtaining image data of a frame rate that matches the display frequency (e.g., 120 Hz) of the display panel 209.
[0114] Furthermore, the image quality improvement processing unit 208c outputs the image data output from the display frequency conversion unit 208b as is. In other words, the image quality improvement processing unit 208c does not perform any processing on the image data output from the display frequency conversion unit 208b. Furthermore, the display RGB conversion unit 208d performs color gamut conversion processing on the image data output from the image quality improvement processing unit 208c to set the color temperature (white point) to 6500K, thereby obtaining image data for display.
[0115] 1-5-2. Display operation for received image data in television receiver The display operation in the television receiver 200 shown in Fig. 13 will be described. First, the operation will be described when the image data (Video) obtained by the HDMI receiver 201 is image data with a frame rate of 60 Hz, which is image information of video material.
[0116] In this case, the control unit 202 determines that the image data (Video) obtained by the HDMI receiver 201 has not been pulled down based on the fact that there is no pull-down infoframe (Pull-down IF), that there is a pull-down infoframe (Pull-down IF) but N=M=1 (see FIG. 11), or that there is a pull-down infoframe (Pull-down IF) but count s=0 (see FIG. 12).
[0117] The control unit 202 switches the changeover switches 204 and 206 to the a side based on the determination that the image data (Video) obtained by the above-mentioned HDMI receiver 201 has not been pulled down. As a result, a 60 Hz vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is supplied to the image conversion unit 207 and the display control unit 208, and the image conversion unit 207 and the display control unit 208 process the image data (Video) with a frame rate of 60 Hz, which is image information of video material obtained by the HDMI receiver 201, to generate image data for display, and an image is displayed on the display panel 209 based on the image data for display.
[0118] In this case, the control unit 202 controls the operations of the image conversion unit 207 and the display control unit 208 based on the AVI infoframe. The display frequency conversion unit 208b, the image quality improvement processing unit 208c, and the display RGB conversion unit 208, which constitute the display control unit 208, perform video processing (see FIG. 15 ) to generate display image data.
[0119] Next, we will explain the operation when the image data (Video) obtained by the HDMI receiver 201 is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material).
[0120] In this case, the control unit 202 determines that the image data (Video) obtained by the HDMI receiver 201 has been pulled down based on the presence of a pull-down infoframe (Pull-down IF), N=M=1, or count s=0.
[0121] Furthermore, the control unit 202 sets the pull-down ratio N:M (N=2, M=5) included in the pull-down infoframe (Pull-down IF) in the N / M frequency dividing unit 203. As a result, the N / M frequency dividing unit 203 divides the 60 Hz vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 by 2 / 5, and obtains a 24 Hz vertical synchronization signal (Vsync').
[0122] Furthermore, the control unit 202 controls the frame extraction process in the frame extraction unit 205 so as to extract image data of a frame corresponding to a change in the count s constituting the identifier in the repeat sequence included in the pull-down infoframe (Pull-down IF). As a result, the frame extraction unit 205 obtains image data with a frame rate of 24 Hz, which is image information of film material (movie material) before pull-down, from the image data (Video) obtained by the HDMI receiver 201.
[0123] Furthermore, the control unit 202 switches the changeover switches 204 and 206 to the b side based on the determination that the image data (Video) obtained by the above-mentioned HDMI receiver 201 has been pulled down. As a result, the 24 Hz vertical synchronization signal (Vsync') obtained by the N / M frequency dividing unit 203 is supplied to the image conversion unit 207 and the display control unit 208, and the image conversion unit 207 and the display control unit 208 process the image data with a frame rate of 24 Hz, which is image information of film material (movie material), obtained by the frame extraction unit 205, to generate image data for display, and an image is displayed on the display panel 209 based on the image data for display.
[0124] In this case, the control unit 202 controls the operations of the image conversion unit 207 and the display control unit 208 based on the AVI infoframe. The display frequency conversion unit 208b, the image quality improvement processing unit 208c, and the display RGB conversion unit 208, which constitute the display control unit 208, perform film (movie) processing (see FIG. 15 ) to generate display image data.
[0125] Also, the operation will be described when the image data (Video) obtained by the HDMI receiver 201 is image data with a frame rate of 24 Hz, which is image information of film material (movie material).
[0126] In this case, the control unit 202 determines that the image data (Video) obtained by the HDMI receiver 201 has not been pulled down, for example, based on the absence of a pull-down infoframe (Pull-down IF).
[0127] The control unit 202 switches the changeover switches 204 and 206 to the a side based on the determination that the image data (Video) obtained by the HDMI receiver 201 described above has not been pulled down. As a result, a 24 Hz vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is supplied to the image conversion unit 207 and the display control unit 208, and the image conversion unit 207 and the display control unit 208 process the image data (Video) with a frame rate of 24 Hz, which is image information of film material (movie material), obtained by the HDMI receiver 201, to generate image data for display, and an image is displayed on the display panel 209 based on the image data for display.
[0128] In this case, the control unit 202 controls the operations of the image conversion unit 207 and the display control unit 208 based on the AVI infoframe. The display frequency conversion unit 208b, the image quality improvement processing unit 208c, and the display RGB conversion unit 208, which constitute the display control unit 208, perform film (movie) processing (see FIG. 15 ) to generate display image data.
[0129] 1-5-3. Control Processing for N / M Dividing Section and Frame Extracting Section The flowchart in FIG. 16 shows an example of control processing for each frame by the control section 202 for the N / M dividing section 203 and frame extracting section 205.
[0130] This example also corresponds to the case where a pull-down infoframe (Pull-down IF) is added to image data that has not been pulled down. Note that the pull-down infoframe (Pull-down IF) in this case also includes one that makes it possible to determine that the image data has not been pulled down by setting count s=0 (see FIG. 13).
[0131] First, the control unit 202 starts processing in step ST1. Next, the control unit 202 determines whether a pull-down infoframe (Pull-down IF) is present in step ST2. If a pull-down infoframe (Pull-down IF) is not present, the control unit 202 determines that the image data is not pulled down, turns off the operation of the N / M frequency divider 203 and the frame extractor 205 in step ST3, and then ends processing in step ST4.
[0132] If a pull-down infoframe (Pull-down IF) is present in step ST2, the control unit 202 determines in step ST5 whether N = M = 1. If N = M = 1, the control unit 202 determines that the image data is not pulled down, turns off the operations of the N / M frequency divider 203 and the frame extractor 205 in step ST3, and then ends the process in step ST4.
[0133] If N=M=1 is not true in step ST5, the control unit 202 sets N and M in the N / M frequency dividing unit 203 and causes it to perform frequency division in step ST6. Next, the control unit 202 determines whether the count s=0 in step ST7. If s=0, the control unit 202 determines that the image data is not pulled down, turns off the operations of the N / M frequency dividing unit 203 and the frame extraction unit 205 in step ST3, and then ends the process in step ST4.
[0134] If s=0 is not established in step ST7, the control unit 202 determines in step ST8 whether or not both the count s and the count b are 1. If s=b=1, the control unit 202 synchronizes the N / M frequency dividing unit 203 in step ST9. This allows the phase of the vertical synchronization signal (Vsync') acquired by the N / M frequency dividing unit 203 to be aligned with the frame phase of the image data (image information) before pull-down, enabling good synchronization with, for example, audio.
[0135] After processing step ST9, the control unit 202 proceeds to processing step ST10. If s=b=1 is not established in step ST8, the control unit 202 immediately proceeds to processing step ST10. In step ST10, the control unit 202 determines whether the count s has changed from the previous frame. If the count s has changed, the control unit 202 causes the frame extraction unit 205 to extract a frame at frame 205 in step ST11. This allows the frame extraction unit 205 to extract a frame of image data (image information) before pull-down.
[0136] After the process of step ST11, the control section 202 ends the process in step ST4. If the count s has not changed in step ST10, the control section 202 immediately ends the process in step ST4.
[0137] In the control process of the flowchart shown in Fig. 16, if a pull-down infoframe (Pull-down IF) that makes it possible to determine that the image data is not pulled down by setting count s = 0 is added to the image data that is not pulled down (see Fig. 12), in step ST6, the control unit 202 sets N and M in the pull-down ratio N:M of the image data that is being pulled down to the N / M frequency dividing unit 203 and causes the frequency dividing operation. This makes it possible to make preparations in advance for the pulled-down image data that is to be switched later, such as preparations for creating a vertical synchronization signal (Vsync) required for image conversion processing and display control processing.
[0138] The flowchart in Fig. 17 shows another example of the control process for each frame by the control unit 202 with respect to the N / M frequency dividing unit 203 and the frame extraction unit 205. In Fig. 17, steps corresponding to those in Fig. 16 are denoted by the same reference numerals. In the control process in the flowchart in Fig. 17, the processes of steps ST8 and ST9 in the control process in the flowchart shown in Fig. 16 are omitted, and synchronization of the N / M frequency dividing unit 203 is not performed when s = b = 1.
[0139] The flowchart of FIG. 18 illustrates another example of frame-by-frame control processing by the control unit 202 for the N / M frequency divider 203 and the frame extractor 205. In FIG. 18, steps corresponding to those in FIG. 16 are denoted by the same reference numerals. This example also accommodates the case where a pull-down infoframe (Pull-down IF) is added to image data that has not been pulled down. However, in this example, the pull-down infoframe (Pull-down IF) does not include the one that allows determination that the image data has not been pulled down by setting count s=0 (see FIG. 12). The control processing of the flowchart of FIG. 18 does not include step ST7 in the control processing of the flowchart of FIG. 16, and does not determine that the image data has not been pulled down by setting count s=0.
[0140] The flowchart of Fig. 19 shows yet another example of the control process for each frame by the control unit 202 with respect to the N / M frequency dividing unit 203 and the frame extraction unit 205. In Fig. 19, steps corresponding to those in Fig. 18 are designated by the same reference numerals. In the control process of the flowchart of Fig. 19, the processes of steps ST8 and ST9 in the control process of the flowchart shown in Fig. 18 are omitted, and synchronization of the N / M frequency dividing unit 203 is not performed when s = b = 1.
[0141] 20 is a schematic diagram showing the processing in the television receiver 200 (see FIG. 13), which is a sink device, when image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), is sent from the media player 100 (see FIG. 4), which is a source device. In the figure, the values of x and y assigned to the pulled-down image data are counts s and b that constitute the identifier within the repeat sequence.
[0142] In the television receiver 200, frame extraction is performed from the pulled-down image data obtained by the pull-down process based on the count s, that is, at the timing when the count s changes, to extract each frame of the image data before pull-down. Next, each frame of the extracted image data before pull-down is aligned in accordance with the timing of the 24 Hz vertical synchronization signal (Vsync') obtained by the N / M frequency divider 203. Next, the aligned image data with a frame rate of 24 Hz is subjected to frame multiplexing processing to generate image data (image data for display) with a frame rate of 120 Hz that matches the display frequency (120 Hz) of the display panel 209.
[0143] 1-6. Other Configuration Examples of Television Receiver Figure 21 shows another configuration example of a television receiver 200. In Figure 21, parts corresponding to those in Figure 13 are designated by the same reference numerals. In addition to an HDMI receiver 201, the television receiver 200 has a control unit 202, an N / M frequency dividing unit 203, a selector switch 204, a frame extraction unit 205, a selector switch 206, an image conversion unit 207, a display control unit 208, a display panel 209, and adders 210 and 211.
[0144] The control unit 202 controls the operation of each unit of the television receiver 200. The HDMI receiver 201 obtains, in addition to image data (Video), a vertical synchronization signal (Vsync) corresponding to this image data (Video), and audio data (Audio) and auxiliary data (Auxiliary) accompanying this image data (Video) from the TMDS transmission data received from the media player 100. Although not shown in the figure, audio output processing is performed based on the audio data (Audio) obtained by the HDMI receiver 201.
[0145] The auxiliary data (Auxiliary) obtained by the HDMI receiver 201 is supplied to the control unit 202. As described above, the auxiliary data (Auxiliary) includes an AVI infoframe, and at least, when the image data (Video) is image data (image information) with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), the auxiliary data includes a pull-down infoframe (Pull-down IF) having pull-down information associated with each frame of the image data (image information).
[0146] Therefore, the control unit 202 acquires a video format identification code from the AVI infoframe and recognizes the picture frame and frame rate of the image data (Video). The control unit 202 also recognizes video-related information, such as the color space of the image data (Video), from the AVI infoframe. The control unit 202 also recognizes from the pulldown infoframe that the image data (Video) is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material), and further recognizes the pulldown ratio N:M and the counts s and b that constitute the identifier within the repeat sequence.
[0147] The changeover switch 204 switches between supplying the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 to the adder 210 and supplying it to the N / M frequency divider 203. In this case, the vertical synchronization signal (Vsync) is input to a movable terminal of the changeover switch 204, and when the image data (Video) is not pulled down, the vertical synchronization signal (Vsync) is supplied to the adder 210 from the fixed terminal on the a side, and when the image data (Video) is pulled down, the vertical synchronization signal (Vsync) is supplied to the N / M frequency divider 203 from the fixed terminal on the b side.
[0148] The control unit 202 sets N and M in the N / M frequency divider unit 203 based on the pulldown ratio N:M recognized from the pulldown infoframe. The N / M frequency divider unit 203 divides the vertical synchronization signal (Vsync) supplied from the fixed terminal on the b side of the changeover switch 204, i.e., the vertical synchronization signal (Vsync) at the frame rate of the image data after pulldown, to obtain a vertical synchronization signal (Vsync') at the frame rate of the image data before pulldown. The vertical synchronization signal (Vsync') obtained by the N / M frequency divider unit 203 is supplied to the adder 210.
[0149] In this case, if the image data (Video) is image data (image information) with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information of film material (movie material), the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is 60 Hz, but the vertical synchronization signal (Vsync') obtained by the N / M frequency divider unit 203 in this way becomes 24 Hz.
[0150] As described above, when the image data (Video) is not pulled down, a vertical synchronization signal (Vsync) is supplied from the fixed terminal on the a side of the changeover switch 204 to the adder 210. Therefore, when the image data (Video) is not pulled down, the vertical synchronization signal (Vsync) obtained by the HDMI receiver 201 is output from the adder 210. Also, when the image data (Video) is pulled down as described above, a vertical synchronization signal (Vsync') is supplied from the N / M frequency divider 203 to the adder 210. Therefore, when the image data (Video) is pulled down, the vertical synchronization signal (Vsync') obtained by the N / M frequency divider 203 is output from the adder 210.
[0151] The changeover switch 206 switches between supplying the image data (Video) obtained by the HDMI receiver 201 to the adder 211 and supplying it to the frame extraction unit 205. In this case, the image data (Video) is input to a movable terminal of the changeover switch 206, and when the image data (Video) is not pulled down, the image data (Video) is supplied to the adder 211 from the fixed terminal on the a side, and when the image data (Video) is not pulled down, the image data (Video) is supplied to the frame extraction unit 205 from the fixed terminal on the b side.
[0152] The frame extraction unit 205 extracts, from the image data (Video), image data of frames corresponding to changes in the count s that constitute the identifier within the repeat sequence. The image data of each frame extracted in this manner corresponds to the image data of each frame before pull-down. For example, if the image data (Video) is image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material), the frame extraction unit 205 obtains image data with a frame rate of 24 Hz that corresponds to the image data of each frame before pull-down. The image data obtained by the frame extraction unit 205 is supplied to an adder 211.
[0153] As described above, when the image data (Video) is not pulled down, the image data (Video) is supplied from the fixed terminal on the a side of the changeover switch 204 to the adder 211. Therefore, when the image data (Video) is not pulled down, the image data (Video) obtained by the HDMI receiver 201 is output from the adder 211. Also, when the image data (Video) is pulled down as described above, the image data obtained by the frame extraction unit 205 is supplied to the adder 211. Therefore, when the image data (Video) is pulled down, the image data obtained by the frame extraction unit 205 is output from the adder 211.
[0154] The image conversion unit 207 performs processing on the image data output from the adder 211 in synchronization with the vertical synchronization signal output from the adder 210, independent of the display panel 209, etc. The display control unit 208 performs processing on the image data after processing by the image conversion unit 207 in synchronization with the vertical synchronization signal output from the adder 210, dependent on the display panel 209, etc. The display panel 209 is configured, for example, with a liquid crystal display panel or an organic EL panel, etc., and displays an image based on the image data after processing by the display control unit 208 (image data for display).
[0155] As described above, in the image processing system 10 shown in FIG. 1, pull-down information is associated with each frame of image data (image information) transmitted from the media player 100 to the television receiver 200, and the television receiver 200 can easily obtain the pull-down information and use it effectively in processing the image data (image information), thereby significantly reducing the processing load.
[0156] 16 to 19 can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, the programs that make up the software are installed from a recording medium into a computer that is built into dedicated hardware, or into, for example, a general-purpose computer that can execute various functions by installing various programs.
[0157] 22 is a block diagram showing an example of the hardware configuration of a computer 600. The computer 600 has a CPU 601, a ROM 602, a RAM 603, a bus 604, an input / output interface 605, an input unit 606, an output unit 607, a storage unit 608, a drive 609, a connection port 610, and a communication unit 611. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the computer 600 may further include components other than those shown here.
[0158] The CPU 601 functions as, for example, an arithmetic processing device or a control device, and controls the overall operation or part of the operation of each component based on various programs recorded in the ROM 602 , the RAM 603 , the storage unit 608 , or the removable recording medium 701 .
[0159] The ROM 602 is a means for storing programs to be read into the CPU 601, data to be used for calculations, etc. The RAM 603 temporarily or permanently stores, for example, the programs to be read into the CPU 601 and various parameters that change as appropriate when the programs are executed.
[0160] The CPU 601, ROM 602, and RAM 603 are connected to one another via a bus 604. On the other hand, various components are connected to the bus 604 via an input / output interface 605.
[0161] The input unit 606 may be, for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, etc. Furthermore, the input unit 606 may also be a remote controller (hereinafter referred to as a remote control) that is capable of transmitting control signals using infrared rays or other radio waves.
[0162] The output unit 607 is a device capable of visually or audibly notifying the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile.
[0163] The storage unit 608 is a device for storing various types of data. For example, the storage unit 608 may be a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device.
[0164] The drive 609 is a device that reads information recorded on a removable recording medium 701 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 701 .
[0165] The removable recording medium 701 is, for example, a DVD, a Blu-ray (registered trademark) disc, an HD DVD, various semiconductor storage media, etc. Of course, the removable recording medium 701 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.
[0166] The connection port 610 is a port for connecting an external device 502, such as a Universal Serial Bus (USB) port, a High-Definition Multimedia Interface (HDMI) port, an IEEE 1394 port, a Small Computer System Interface (SCSI), an RS-232C port, or an optical audio terminal. The external device 702 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.
[0167] The communication unit 611 is a communication device for connecting to the network 703, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.
[0168] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.
[0169] 2. Modifications 2-1. Other Examples of Pulldown Note that, in the above-described embodiment, pulldown from 24 Hz to 60 Hz has been described, but the pulldown to which the present technology can be applied is not limited to this. Figures 23(a) to 23(c) show examples of pulldown.
[0170] FIG. 23(a) shows so-called 2-3 pulldown from 24 Hz to 60 Hz, similar to FIG. 5(a) above, with a pulldown ratio of 2:5. FIG. 23(b) shows pulldown from 48 Hz to 60 Hz with a pulldown ratio of 4:5. FIG. 23(c) shows pulldown from 50 Hz to 60 Hz with a pulldown ratio of 5:6. Note that in these figures, frames marked with an "*" after pulldown are identical to the previous frame (duplicate frame). This also applies to the figures shown subsequently.
[0171] Figures 24(a) to (c) show examples of pulldown from 48 Hz to 60 Hz. In this pulldown from 48 Hz to 60 Hz, every four frames before pulldown are converted into five frames after pulldown. In the pulldown shown in Figure 24(a), the overlapping frame is positioned at the end (the fifth frame) of the five frames after pulldown. In the pulldown shown in Figure 24(b), the overlapping frame is positioned not at the end but somewhere in the middle (the fourth frame in this case). In the pulldown shown in Figure 24(c), the position of the overlapping frame is variable among the five frames after pulldown.
[0172] Although detailed description will be omitted, there are multiple types of pulldown from 50 Hz to 60 Hz, just like pulldown from 24 Hz to 60 Hz and pulldown from 48 Hz to 60 Hz.
[0173] Figures 25(a) to (c) show an example of the pulldown ratio N:M included in a pulldown infoframe, for example, which is associated with each frame in pulldown with multiple pulldown ratios, and the counts s and b that constitute the identifier in the repeat sequence.
[0174] 25(a) is the same as the above-mentioned Fig. 8 and shows an example of pulldown from 24 Hz to 60 Hz. In this case, the ratio of the frame rate of the source frame (24 Hz) to the frame rate of the base frame (60 Hz) is 2:5, i.e., the pulldown ratio is 2:5, so N=2 and M=5.
[0175] The repeat sequence is a five-frame cycle from the first frame to the fifth frame, with counts s and b being "1, 1" for the first frame, "1, 2" for the second frame, "2, 3" for the third frame, "2, 4" for the fourth frame, and "2, 5" for the fifth frame. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 2 (N), and frames with the same count s value are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 5 (M). When s = b = 1, the frame phases before and after pulldown match.
[0176] 25(b) shows an example of pulldown from 48 Hz to 60 Hz. In this case, the ratio of the frame rate of the source frame (48 Hz) to the frame rate of the base frame (60 Hz) is 4:5, i.e., the pulldown ratio is 4:5, so N=4 and M=5.
[0177] The repeat sequence is a five-frame cycle from the first frame to the fifth frame, with counts s and b being "1, 1" for the first frame, "2, 2" for the second frame, "3, 3" for the third frame, "4, 4" for the fourth frame, and "4, 5" for the fifth frame. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 4 (N), and frames with the same count s value are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 5 (M). When s = b = 1, the frame phases before and after pulldown match.
[0178] 25(c) shows an example of pulldown from 50 Hz to 60 Hz. In this case, the ratio of the frame rate of the source frame (50 Hz) to the frame rate of the base frame (60 Hz) is 5:6, i.e., the pulldown ratio is 5:6, so N=5 and M=6.
[0179] The repeat sequence is a six-frame cycle from frame 1 to frame 6, with counts s and b being "1, 1" for frame 1, "2, 2" for frame 2, "3, 3" for frame 3, "4, 4" for frame 4, "5, 5" for frame 5, and "5, 6" for frame 6. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 5 (N), and frames with the same count s value are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 6 (M). When s = b = 1, the frame phases before and after pulldown match.
[0180] "2-2. Case of 24 Hz / 48 Hz Mixed Content" The case of 24 Hz / 48 Hz mixed content will be described. For example, film material (movie material) may contain a mixture of image data (image information) with a frame rate of 24 Hz and image data (image information) with a frame rate of 48 Hz. For example, 24 Hz is used in lyrical scenes to create a cinematic feel, and 48 Hz is used in action scenes and other scenes where movement needs to be emphasized.
[0181] 26(a) and (b) show examples of pulldown information associated with each frame when image data (image information) with a frame rate of 24 Hz is pulled down from 24 Hz to 60 Hz (first pulldown) and image data (image information) with a frame rate of 48 Hz is pulled down from 48 Hz to 60 Hz (second pulldown). Fig. 26(a) shows a state of switching from image data with the first pulldown to image data with the second pulldown, and Fig. 26(b) shows a state of switching in the reverse direction.
[0182] The first pulldown portion will now be described. In this case, the ratio of the frame rate of the source frame (24 Hz) to the frame rate of the base frame (60 Hz) is 2:5, i.e., the pulldown ratio is 2:5, so N = 2 and M = 5. The repeat sequence is a five-frame cycle from frame 1 to frame 5, with counts s and b being "1, 1" for frame 1, "1, 2" for frame 2, "2, 3" for frame 3, "2, 4" for frame 4, and "2, 5" for frame 5. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 2 (N), and frames with the same value of this count s are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 5 (M). When s = b = 1, the frame phases before and after pulldown are the same.
[0183] Next, we will explain the second pulldown portion. In this case, the ratio of the frame rate of the source frame (48 Hz) to the frame rate of the base frame (60 Hz) is 4:5, i.e., the pulldown ratio is 4:5, so N = 4 and M = 5. The repeat sequence is a five-frame cycle from frame 1 to frame 5, with counts s and b being "1, 1" for frame 1, "2, 2" for frame 2, "3, 3" for frame 3, "4, 4" for frame 4, and "4, 5" for frame 5. The count s of the source frame corresponding to the frame before pulldown repeats from 1 to 4 (N), and frames with the same count s value are duplicate (retransmitted) frames. The count b of the base frame corresponding to the frame after pulldown repeats from 1 to 5 (M). When s = b = 1, the frame phases before and after pulldown are the same.
[0184] 27(a) and (b) show examples of pull-down information associated with each frame when image data (image information) with a frame rate of 24 Hz is pulled down from 24 Hz to 48 Hz, and image data (image information) with a frame rate of 48 Hz is not pulled down. Fig. 27(a) shows a state of switching from pull-down image data to non-pull-down image data, and Fig. 27(b) shows a state of switching in the reverse direction.
[0185] The pulldown portion will now be explained. In this case, the ratio of the frame rate of the source frame (24 Hz) to the frame rate of the base frame (48 Hz) is 1:2, i.e., the pulldown ratio is 1:2, so N = 1 and M = 2. The repeat sequence has a two-frame cycle from the first frame to the second frame, and the counts s and b are "1, 1" for the first frame and "1, 2" for the second frame. On the receiving side, by extracting the frame where s = b = 1, it is possible to obtain image data (image information) with a frame rate of 24 Hz before pulldown. Furthermore, when s = b = 1, the frame phases before and after pulldown are the same.
[0186] Next, we will explain the non-pull-down portion. In this case, count s is fixed to 0. This allows the receiving side to determine that the received image data (image information) is not pull-down, based on the fact that count s = 0.
[0187] In this case, the ratio between the frame rate of the source frame and the frame rate of the base frame is 1:1, i.e., the pulldown ratio is 1:1, but the pulldown ratio N:M indicates a pulldown ratio of 1:2 in the pulldown portion, where N = 1 and M = 2. This makes it possible to make preparations in advance for the image data of the pulldown portion, such as preparations for creating a vertical synchronization signal required for image conversion processing and display control processing.
[0188] In the example shown in Figures 27(a) and (b), the count s is fixed to 0 for the non-pull-down portion. However, it is also possible to set s = b = N = M = 1, as in the example shown in Figures 11(a) and (b) above.
[0189] "2-3. Other Examples of Pull-Down Information" In the above-described embodiment, the pull-down information associated with image data (image information) for each frame is the pull-down ratio (N:M) and an identifier (s, b) within the repeat sequence. However, other examples of pull-down information associated with image data (image information) for each frame are also possible.
[0190] 28(a) and (b) show examples in which pull-down information associated with image data (image information) for each frame is a duplicate flag that constitutes flag information indicating whether the image data overlaps with the immediately preceding frame. Fig. 28(a) shows a state of switching from 60 Hz frame rate image data obtained by pulling down 24 Hz frame rate image data, which is image information of film material (movie material), to 60 Hz frame rate image data, which is image information of video material that has not been pulled down, and Fig. 28(b) shows a state of switching in the reverse direction.
[0191] In this case, the duplicate flag is set to "1" for frames that overlap with the immediately preceding frame, and to "0" for frames that do not overlap with the immediately preceding frame. Therefore, on the receiving side, based on this duplicate flag, it is possible to determine which frames overlap with the immediately preceding frame without performing correlation detection or the like, thereby reducing the processing load on the pulled-down image data.
[0192] 29(a) and (b) show examples in which the pull-down information associated with image data (image information) for each frame is a pull-down flag that constitutes flag information indicating whether or not pull-down has been performed. Fig. 29(a) shows a state of switching from 60 Hz frame rate image data obtained by pulling down 24 Hz frame rate image data, which is image information of film material (movie material), to 60 Hz frame rate image data, which is image information of video material that has not been pulled down, and Fig. 29(b) shows a state of switching in the reverse direction.
[0193] In this case, the pulldown flag is set to "1" for each frame of image data with a frame rate of 60 Hz obtained by pulling down image data with a frame rate of 24 Hz, which is image information from film material (movie material), and is set to "0" for each frame of image data with a frame rate of 60 Hz, which is image information from video material that has not been pulled down. Therefore, on the receiving side, based on this pulldown flag, the start point of pulldown and the like can be detected without performing correlation detection, etc., and it becomes possible to reduce the processing load on the pulled-down image data.
[0194] 30(a) and (b) show examples in which the pull-down information associated with image data (image information) for each frame is both the above-mentioned duplicate flag and pull-down flag. Fig. 30(a) shows a state of switching from 60 Hz frame rate image data obtained by pulling down 24 Hz frame rate image data, which is image information of film material (movie material), to 60 Hz frame rate image data, which is image information of video material that has not been pulled down, and Fig. 30(b) shows a state of switching in the reverse direction.
[0195] In this case, the duplicate flag is set to "1" for frames that overlap with the immediately preceding frame and to "0" for frames that do not overlap with the immediately preceding frame. The pull-down flag is set to "1" for each frame of 60-Hz frame rate image data obtained by pulling down 24-Hz frame rate image data, which is image information from film material (movie material), and is set to "0" for each frame of 60-Hz frame rate image data, which is image information from video material that has not been pulled down. Therefore, on the receiving side, based on the duplicate flag, it is possible to determine frames that overlap with the immediately preceding frame without performing correlation detection, etc., and based on the pull-down flag, it is possible to detect the start point of pull-down, etc., without performing correlation detection, etc., thereby reducing the processing load on the pulled-down image data.
[0196] 31(a) and (b) show an example of the data structure of a newly defined video frame structure infoframe (Video frame structure IF) packet including both a duplicate flag and a pull-down flag, where Fig. 31(a) shows the packet header (Pull-down IF Packet Header) and Fig. 31(b) shows the packet contents (Pull-down IF Packet Contents).
[0197] First, the packet header (Video frame structure IF Packet Header) shown in Fig. 31(a) will be described. Byte HB0 defines "Packet Type," which indicates the type of packet. Here, this indicates a Video Frame Structure Info Frame packet. Byte HB1 describes the version of the packet data definition. Byte HB2 describes information indicating the packet length. Here, this is set to "0x01."
[0198] Next, the packet contents (Pull-down IF Packet Contents) shown in Fig. 31(b) will be described. A pull-down flag (PDN: Pull-down IF) is written in the first bit of byte PB1, and a duplicate flag (DPL: duplicate flag) is written in the zeroth bit of byte PB1.
[0199] 31(c) shows an example of the data structure of a packet header (Video frame structure IF Packet Header) containing only a duplicate flag. In this case, the duplicate flag (DPL) is written in bit 0 of byte PB1.
[0200] 31(d) shows an example of the data structure of a packet header (Video frame structure IF Packet Header) containing only a pull-down flag, in which the pull-down flag (PDN: Pull-down IF) is written in the first bit of byte PB1.
[0201] FIG. 32(a) shows the header of an EMP (Extended Metadata Packet), which is a packet having a video frame structure EM (Video Frame Structure Extended Metadata) including a duplicate flag and a pull-down flag, and FIG. 32(b) shows the contents of the EMP.
[0202] The video frame structure EM is small in size and is transmitted in a single packet. Therefore, the fields of the header (EMP Header) are set to the following values: Last=1, First=1, Sequence_Index=0.
[0203] In addition, since the video frame structure EM is video frame synchronized in relation to the video format, each field of the contents (EMP Contents) is set to the following values: Sync=1 VFR=1 AFR=0 DS_Type=2: Unique EM Data Set End=1, New=1: Transmitted in a single packet Organization_ID=1: Video frame structure EM is EM specified by HDMI Data_Set_Tag =xxxx: Identification number for the Video frame structure EM assigned by HDMI Data_Set_Length =1: Pull-down EM uses 1 byte of MD0
[0204] Figure 32(c) shows the MD (MetaData) content, i.e., the video frame structure EM, located in bytes PB7-PB27 of the content (EMP Contents). Although a detailed description is omitted, byte MD0 describes a duplicate flag (DPL) and a pull-down flag (PDN), similar to byte PB1 in the content of the video frame structure infoframe packet shown in Figure 31(b) above.
[0205] "2-4. Others" In the above-described embodiment, pull-down information is associated with image data (image information) for each frame by inserting an infoframe or EMP having pull-down information into the blanking period of the image data (image information), but the association method is not limited to this. For example, a method using a table that associates the frame number of each frame of image data (image information) with pull-down information can also be considered.
[0206] In the above-described embodiment, an example was shown in which image data (image information) in which pull-down information is associated with each frame is transmitted to the receiving side (display side) via an HDMI transmission path, but the transmission path is not limited to an HDMI transmission path and may be other wired or wireless transmission paths. Furthermore, the transmission path may be a broadcast path or a communication path.
[0207] In the above-described embodiment, an example was shown in which image data (image information) in which pull-down information is associated with each frame is transmitted to a receiving side (display side) via an HDMI transmission path. However, the present technology can also be applied to a system in which image data (image information) in which pull-down information is associated with each frame is written to a recording medium and the image data (image information) in which pull-down information is associated with each frame is supplied to a display side using the recording medium. The present technology can also be applied to a system in which image data (image information) in which pull-down information is associated with each frame is stored in a server on a network and the display side acquires image data (image information) in which pull-down information is associated with each frame from the server.
[0208] Furthermore, in the above-described embodiment, an example was given in which the source device was a media player, but the source device is not limited to a media player and may be any other device with similar functions, such as a BD player, an Internet-compatible streaming device (network media player), a set-top box, etc. Similarly, the sink device is not limited to a television receiver and may be any device with similar display functions.
[0209] Furthermore, while the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.
[0210] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.
[0211] The technology may also be configured as follows: (1) An image processing device including an information associating unit that associates pull-down information with image information for each frame. (2) The image processing device according to (1), wherein the pull-down information includes a pull-down ratio and an identifier within a repeat sequence. (3) The image processing device according to (2), wherein the pull-down ratio is set to 1:1 if the image information is not pulled down. (4) The image processing device according to (2), wherein the identifier is composed of a combination of a source frame count corresponding to a frame before pull-down and a base frame count corresponding to a frame after pull-down. (5) The image processing device according to (4), wherein the source frame count is fixed to 0 if the image information is not pulled down. (6) The image processing device according to (1), wherein the pull-down information includes flag information indicating whether or not the image information overlaps with the immediately preceding frame. (7) The image processing device according to (1), wherein the pull-down information includes flag information indicating whether or not the image information is pulled down. (8) The image processing device according to any one of (1) to (7), wherein the information associating unit inserts the pull-down information into a blanking period of image information for each frame. (9) The image processing device according to (8), wherein the information associating unit inserts the pull-down information into the blanking period as IF (InfoFrame) information. (10) The image processing device according to (8), wherein the information associating unit inserts the pull-down information into the blanking period as EMP (Extended Metadata Packet) information. (11) The image processing device according to any one of (1) to (10), further comprising a transmitting unit that transmits the image information associated with the pull-down information for each frame to an external device via a transmission path. (12) The image processing device according to any one of (1) to (10), further comprising a processing unit that extracts second image information of a frame before pull-down from first image information based on the pull-down information associated with the first image information for each frame, to obtain image information for display. (13) The image processing device according to (12), wherein the pull-down information includes a pull-down ratio and an identifier within a repeat sequence.(14) The image processing device according to (13), wherein the processing unit acquires second image information of the frame before pull-down based on an identifier in the repetitive sequence, sets a frequency division ratio of a frequency divider based on the pull-down ratio to acquire a synchronization signal corresponding to the frame rate of the frame before pull-down, and processes the acquired second image information of the frame before pull-down based on the acquired synchronization signal to acquire the image information for display. (15) The image processing device according to (14), wherein the identifier is composed of a combination of a source frame count corresponding to the frame before pull-down and a base frame count corresponding to the frame after pull-down, and the processing unit acquires the second image information of the frame before pull-down by extracting image information of a frame corresponding to a change in the source frame count from the first image information. (16) The image processing device according to (14) or (15), wherein the identifier is composed of a combination of a source frame count corresponding to the frame before pull-down and a base frame count corresponding to the frame after pull-down, and the processing unit synchronizes the frequency divider at the timing of a frame when both the source frame count and the base frame count become 1. (17) The image processing device according to any one of (12) to (16), wherein the processing unit determines whether the first image information has been pulled down based on the pull-down information, and if the first image information has been pulled down, obtains second image information of a frame before the pull-down based on the pull-down information from the first image information to obtain image information for display. (18) The image processing device according to (17), wherein the pull-down information includes a pull-down ratio, and if the pull-down ratio is 1:1, the processing unit determines that the first image information has not been pulled down.(19) The image processing device according to (18), wherein the pull-down information includes an identifier within a repeat sequence, the identifier being composed of a combination of a source frame count corresponding to a frame before pull-down and a base frame count corresponding to a frame after pull-down, and the processing unit determines that the first image information has not been pulled down if the source frame count is 0. (20) An image processing system comprising a first image processing device and a second image processing device, wherein the first image processing device comprises an information associating unit that associates pull-down information with first image information for each frame, and the second image processing device comprises a processing unit that obtains second image information of a frame before pull-down from the first image information based on the pull-down information associated with the first image information for each frame, to obtain image information for display.
[0212] 10... Image processing system 100... Media player 101... HDMI transmitter 102... Control unit 103... Pull-down conversion unit 104, 105... Changeover switch 200... Television receiver 201... HDMI receiver 202... Control unit 203... N / M frequency division unit 204, 206... Changeover switch 205... Frame extraction unit 207... Image conversion unit 207a... RGB conversion unit 207b... EOTF processing unit 208... Display control unit 208a... Display frame conversion unit 208b... Display frequency conversion unit 208c... Image quality improvement unit 208d... Display RGB conversion unit 209... Display panel 210, 211... Adder 300... HDMI cable 600... Computer
Claims
1. An image processing device having an information association unit that associates pull-down information with image information for each frame.
2. The image processing device according to claim 1, wherein the pulldown information includes a pulldown ratio and an identifier within a repeat sequence.
3. The image processing device according to claim 2, wherein if the image information is not pull-down information, the pull-down ratio is set to 1:
1.
4. The image processing device according to claim 2, wherein the identifier is composed of a combination of a source frame count corresponding to a frame before pulldown and a base frame count corresponding to a frame after pulldown.
5. The image processing device according to claim 4, wherein the source frame count is fixed to 0 if the image information is not pulled down.
6. The image processing device according to claim 1, wherein the pull-down information includes flag information indicating whether or not the pull-down information overlaps with the immediately preceding frame.
7. The image processing device according to claim 1, wherein the pull-down information includes flag information indicating whether or not the image is pulled down.
8. The image processing device according to claim 1, wherein the information associating section inserts the pull-down information into a blanking period of image information for each frame.
9. The image processing device according to claim 8, wherein the information associating section inserts the pull-down information into the blanking period as IF (InfoFrame) information.
10. The image processing device according to claim 8, wherein the information associating section inserts the pulldown information into the blanking period as EMP (Extended Metadata Packet) information.
11. The image processing device according to claim 1, further comprising a transmission unit that transmits the image information, in which pull-down information is associated with each frame, to an external device via a transmission path.
12. An image processing device comprising a processing unit that extracts second image information of a frame before pulldown from first image information based on pulldown information associated with the first image information for each frame, to obtain image information for display.
13. The image processing device according to claim 12, wherein the pulldown information includes a pulldown ratio and an identifier within a repeat sequence.
14. The image processing device according to claim 13, wherein the processing unit acquires second image information of the frame before pull-down based on an identifier in the repetitive sequence, sets a frequency division ratio of a frequency division unit based on the pull-down ratio to acquire a synchronization signal corresponding to the frame rate of the frame before pull-down, and processes the acquired second image information of the frame before pull-down based on the acquired synchronization signal to acquire the image information for display.
15. The image processing device according to claim 14, wherein the identifier is composed of a combination of a source frame count corresponding to a frame before pull-down and a base frame count corresponding to a frame after pull-down, and the processing unit obtains second image information of the frame before pull-down by extracting image information of a frame corresponding to a change in the source frame count from the first image information.
16. The image processing device according to claim 14, wherein the identifier is composed of a combination of a source frame count corresponding to a frame before pulldown and a base frame count corresponding to a frame after pulldown, and the processing unit synchronizes the frequency divider unit at the timing of a frame where both the source frame count and the base frame count become 1.
17. The image processing device according to claim 12, wherein the processing unit determines whether or not the first image information has been pulled down based on the pull-down information, and if the first image information has been pulled down, obtains second image information of the frame before the pull-down from the first image information based on the pull-down information to obtain image information for display.
18. The image processing device according to claim 17, wherein the pull-down information includes a pull-down ratio, and the processing unit determines that the first image information is not pulled down if the pull-down ratio is 1:
1.
19. The image processing device according to claim 17, wherein the pulldown information includes an identifier in a repeating sequence, the identifier being composed of a combination of a source frame count corresponding to a frame before pulldown and a base frame count corresponding to a frame after pulldown, and the processing unit determines that the first image information has not been pulled down if the source frame count is 0.
20. An image processing system comprising a first image processing device and a second image processing device, wherein the first image processing device has an information associating unit that associates pull-down information with first image information for each frame, and the second image processing device has a processing unit that obtains second image information of a frame before pull-down from the first image information based on the pull-down information associated with the first image information for each frame, to obtain image information for display.
Citation Information
Patent Citations
Picture encoding device / Method
JP2000069479A
Image processor, image processing method, and program
JP2008124741A
Data processing apparatus
WO2006054590A1
