Receiving device and program

Abstract

To suppress deterioration of an image quality at the time of starting reproduction, while reducing delay until the start of the reproduction of a moving image streaming.SOLUTION: A receiving device 1 calculates a first time required for receiving a segment of the maximum video image quality, one or more a second time required for receiving the segment of a video image quality which is not the maximum, and one or more a third time required for receiving the segment of each video image quality which is not the maximum to perform a super-resolution processing before a reproduction of a moving image content. The receiving device comprises: an initial video image quality determination part 13 that in the case where the first time is less than a first threshold value, sets an initial video image quality at a video image quality corresponded to the first time, and in the case where the first time is a first threshold value or more, sets the initial video image quality to the video image quality corresponded to a time which becomes a second threshold value or less of the second time and the third time; a segment acquisition part 14 that acquires the segment of the determined video image quality; a decode part 16 that decodes the segment; and a super-resolution processing part 17 that performs the super-resolution processing in the case where the video image quality corresponding to the third time is determined as the initial video image quality.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a receiving device and a program. [Background technology]

[0002] In recent years, the mainstream streaming video distribution method on the Internet is to use HTTP protocol for streaming distribution by general-purpose web servers. One example of such a streaming distribution method (adaptive streaming) using HTTP protocol is the international standard MPEG-DASH (ISO / IEC23009-1).

[0003] In MPEG-DASH, a Web server is provided with segments, each of which is a stream of video content encoded at one or more video qualities (screen size and bit rate) and divided into files of several to several tens of seconds each, and a manifest file that describes the attributes and URLs of the video content. A receiving device selects an appropriate video quality from the manifest file, taking into consideration the screen size of the receiving device and the state of the network bandwidth of the transmission path, and receives the segments one after another, splicing them together into a single video content and playing it back (for example, see Non-Patent Document 1).

[0004] It is also generally known that in adaptive streaming, delays until playback starts can be reduced by selecting and receiving low-bitrate segments at the initial stage of playback.As another method for reducing delays until playback starts, Patent Document 1 discloses a method for shortening the waiting time until playback starts while preventing playback from being interrupted by measuring the maximum delay time according to the type of network and communication conditions, and acquiring the smallest number of media data among a plurality of media data corresponding to playback section information within a range in which the sum of playback section lengths from the last media data in the playback order exceeds the maximum delay time. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2015-041800 [Non-patent literature]

[0006] [Non-Patent Document 1] “Next-generation video distribution technology "MPEG-DASH" technical overview and trends in standardization and related technologies”, Journal of the Institute of Image Information and Television Engineers, Vol.67, No.2, 2013, pp.109-115 Summary of the Invention [Problem to be solved by the invention]

[0007] However, conventional adaptive streaming has a problem in that the image quality at the start of playback is degraded by selecting low-quality video in order to reduce the delay at the start of playback. Even if the technology described in Patent Document 1 is used, it is not possible to suppress the degradation of image quality at the start of playback.

[0008] The object of the present invention, made in consideration of the above circumstances, is to provide a receiving device and a program capable of reducing the delay until playback starts while suppressing degradation in image quality at the start of playback when streaming video is played. [Means for solving the problem]

[0009] In order to solve the above problem, a receiving device of the present invention is a receiving device that receives segments selected from multiple types of video quality, and calculates a first time required to receive a segment of the highest video quality, one or more second times required to receive a segment of each video quality other than the highest, and one or more third times required to receive a segment of each video quality other than the highest and perform super-resolution processing before playing back video content, and is equipped with an initial video quality determination unit that determines an initial video quality to be the video quality corresponding to the first time if the first time is less than a first threshold, and determines an initial video quality to be the video quality corresponding to the longest time among the second time and the third time that is less than the second threshold if the first time is greater than or equal to the first threshold, a segment acquisition unit that acquires a segment of the video quality determined by the initial video quality determination unit, a decoding unit that decodes the segment acquired by the segment acquisition unit to generate decoded data, and a super-resolution processing unit that performs super-resolution processing on the decoded data when the video quality corresponding to the third time is determined as the initial video quality by the initial video quality determination unit.

[0010] Furthermore, in one embodiment, the first time is the time required for buffering high-quality video, the second time is the time required for buffering low-quality video, and the third time is the sum of the delay time required until playback starts when the low-quality video is subjected to super-resolution processing and the second time, and the initial video quality determination unit determines the initial video quality to be the high-quality video when the first time is less than the first threshold, and determines the initial video quality to be the low-quality video when the first time is equal to or greater than the first threshold, and the super-resolution processing unit performs super-resolution processing when the first time is equal to or greater than the first threshold and the third time is less than the second threshold.

[0011] Furthermore, in one embodiment, a super-resolution processing time management unit is provided that manages a super-resolution processing time list indicating the processing time required for super-resolution processing for each display resolution of the image, and the initial image quality determination unit may determine the delay time during the super-resolution processing based on the super-resolution processing time list.

[0012] Furthermore, in one embodiment, the system includes a manifest acquisition unit that acquires and analyzes a manifest file, generates a segment URL list that associates the playback time of a segment with the URL of the segment, and generates a video quality list that associates the bit rate and resolution of each video quality of the segment, and the initial video quality determination unit calculates the first time and the second time based on the video quality list, and the segment acquisition unit extracts the URL of the segment of the video quality determined by the initial video quality determination unit from the segment URL list, and generates a segment request including the URL.

[0013] In order to achieve the above object, a program according to the present invention causes a computer to function as the receiving device. Effect of the Invention

[0014] According to the present invention, in adaptive streaming, it is possible to reduce the delay until the start of playback and suppress deterioration in image quality at the start of playback. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram showing an outline of a content distribution system including a receiving device according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of the configuration of a receiving device according to an embodiment of the present invention; [Diagram 3] 5 is a flowchart showing a process at the start of playback of a receiving device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0017] 1 shows an overview of a content distribution system including a receiving device according to an embodiment of the present invention. The content distribution system is a system that distributes video content, which is composed of a plurality of segments and a manifest file, for each segment. A receiving device 1 and a distribution server 3 are connected via the Internet. A content generating device 2 and the distribution server 3 may be connected by a dedicated local network, or may be connected via the Internet.

[0018] The receiving device 1 receives segments of video content and a manifest file from the distribution server 3, and performs streaming playback. There are multiple types of video quality for the segments, and the receiving device 1 receives a segment selected from the multiple types of video quality.

[0019] The content generator 2 generates segments of video content and a manifest file, and provides them to the distribution server 3.

[0020] The distribution server 3 is a server that distributes the manifest file and segments of the URL designated by the receiving device 1 to the receiving device 1, and may be, for example, a general web server.

[0021] (Receiving device) Next, a description will be given of the configuration of the receiving device 1. Fig. 2 is a block diagram showing an example of the configuration of the receiving device 1. The receiving device 1 shown in Fig. 2 includes a manifest acquisition unit 11, a super-resolution processing time management unit 12, an initial video quality determination unit 13, a segment acquisition unit 14, a buffer 15, a decoding unit 16, a super-resolution processing unit 17, and a playback unit 18.

[0022] When a program is selected, the manifest acquisition unit 11 generates a manifest request and transmits it to the distribution server 3. Then, as a response to the request, the manifest file of the desired program is received from the distribution server 3. The manifest file is, for example, a file in MPD (Media Presentation Description) format conforming to MPEG-DASH (ISO / IEC23009-1).

[0023] Then, the manifest acquisition unit 11 analyzes the received manifest file, generates a segment URL list in which the playback times of the segments are associated with the URLs of the segments, and outputs the list to the segment acquisition unit 14.

[0024] Furthermore, the manifest acquisition unit 11 generates a video quality list in which the bit rate and resolution of each video quality segment are associated with each other, and outputs the video quality list to the initial video quality determination unit 13. Table 1 shows an example of the video quality list. Note that while Table 1 shows a video quality list in which the video quality of a segment is of two types, low quality and high quality, the video quality of a segment may be of three or more types.

[0025] [Table 1]

[0026] The super-resolution processing time management unit 12 manages a super-resolution processing time list indicating the processing time (super-resolution processing time) required for super-resolution processing for each display resolution of the video in the receiving device 1 (screen resolution of the playback unit 18). The super-resolution processing time is measured in advance. The super-resolution processing time management unit 12 outputs the super-resolution processing time list to the initial video quality determination unit 13. Table 2 shows an example of the super-resolution processing time list. The higher the display resolution, the longer the super-resolution processing time. Note that, although Table 2 shows the super-resolution processing time as a multiple of the actual time, this is not limited to this.

[0027] [Table 2]

[0028] Based on the throughput information indicating the throughput when receiving the video stream and the video quality list input from the manifest acquisition unit 11, the initial video quality determination unit 13 calculates a first time required to receive a segment of the highest video quality and one or more second times required to receive a segment of each video quality other than the highest before playing the video content. The throughput may be a value measured in advance (the throughput at the time of previous playback). Furthermore, based on the super-resolution processing time list input from the super-resolution processing time management unit 12, the initial video quality determination unit 13 calculates one or more third times required to receive a segment of each video quality other than the highest and perform super-resolution processing before playing the video content.

[0029] Then, when the first time is less than the first threshold, the initial video quality determination unit 13 determines the initial video quality at the start of playback to be the video quality corresponding to the first time (i.e., the highest video quality). When the first time is equal to or greater than the first threshold, the initial video quality at the start of playback is determined to be the video quality corresponding to the longest time between the second time and the third time that is equal to or less than the second threshold. Alternatively, when the first time is equal to or greater than the first threshold, the initial video quality at the start of playback is determined to be the video quality corresponding to the time between the second time and the third time that is equal to or less than the second threshold and has the highest resolution. Note that the first threshold and the second threshold may be the same value. Then, the initial video quality determination unit 13 outputs initial video quality information indicating the determined video quality to the segment acquisition unit 14.

[0030] Moreover, when the initial video quality determination unit 13 determines that the initial video quality is the video quality corresponding to the first or second time, it determines not to perform super-resolution processing, and when the initial video quality is determined to be the video quality corresponding to the third time, it determines to perform super-resolution processing. Then, the initial video quality determination unit 13 outputs super-resolution determination information indicating the determination result to the super-resolution processing unit 17.

[0031] For example, when the video quality of a segment is of three types, high quality, medium quality, and low quality, the initial video quality determination unit 13 calculates the time t1 required to receive a high quality segment as the first time, calculates the time t2 required to receive a medium quality segment and the time t3 required to receive a low quality segment as the second time, and calculates the time t4 required to receive a medium quality segment and perform super-resolution processing and the time t5 required to receive a low quality segment and perform super-resolution processing as the third time. If the time t1 is less than the first threshold T1, the initial video quality is determined to be a high quality video. If the time t1 is equal to or greater than the first threshold T1, the initial video quality is determined to be a video quality corresponding to the longest time that is equal to or less than the second threshold T2 among the times t2 to t5. The initial video quality determination unit 13 then outputs initial video quality information indicating the determined initial video quality to the segment acquisition unit 14. In addition, when the initial video quality determination unit 13 determines the initial video quality to be the video quality corresponding to the first time (t1) or the second time (t2, t3), it determines not to perform super-resolution processing, and when the initial video quality is determined to be the video quality corresponding to the third time (t4, t5), it determines to perform super-resolution processing.

[0032] For convenience of explanation, the following will describe in detail the case where the video quality of a segment is of two types, high quality and low quality. The initial video quality determination unit 13 calculates the high-quality video reception time and the low-quality video reception time before playing back video content, based on the throughput information indicating the throughput when receiving a video stream and the video quality list input from the manifest acquisition unit 11. Here, the "high-quality video reception time" refers to the time (first time) required for buffering high-quality video (segment with high video quality). The "low-quality video reception time" refers to the time (second time) required for buffering low-quality video (segment with low video quality). In addition, the initial video quality determination unit 13 calculates the super-resolution processing time before playing back video content, based on the super-resolution processing time list input from the super-resolution processing time management unit 12. Here, "super-resolution processing time" refers to the sum of the delay time required before playback can begin when low-quality video is super-resolution processed (the time required for smooth playback from the start of super-resolution processing to the start of playback) and the low-quality video reception time (third time).

[0033] When the high-quality video reception time is less than the first threshold T1, the initial video quality determination unit 13 determines the initial video quality at the start of playback to be high-quality video and determines not to perform super-resolution processing. When the high-quality video reception time is equal to or greater than the first threshold T1 and the super-resolution processing time is less than the second threshold T2, the initial video quality determination unit 13 determines the initial video quality at the start of playback to be low-quality video and determines to perform super-resolution processing on the low-quality video. When the high-quality video reception time is equal to or greater than the first threshold T1 and the super-resolution processing time is equal to or greater than the second threshold T2, the initial video quality determination unit 13 determines the initial video quality at the start of playback to be low-quality video and determines not to perform super-resolution processing.

[0034] As a specific example, the video quality list includes two types of video quality, a high bit rate of 5 Mbps and a low bit rate of 500 kbps, and the amount of buffer that should be stored in the receiving device 1 before playback starts is 4 seconds. In addition, the throughput is 10 Mbps. Under these conditions, the time required to store 4 seconds in the buffer is 0.2 seconds for a low bit rate (500 kbps) and 2.0 seconds for a high bit rate (5 Mbps). In the example shown in Table 2, if the display resolution is 1920x1080, the super-resolution processing time is 1.2 times the actual time, so 4 seconds of super-resolution processing takes 4.8 seconds. In other words, if playback is started 0.8 seconds after the start of super-resolution processing, playback can be performed without delay, so the delay time during super-resolution processing is 0.8 seconds. In other words, in this case, the low-quality video reception time is 0.2 seconds, the high-quality video reception time is 2.0 seconds, and the super-resolution processing time is 1.0 seconds, which is the sum of 0.2 seconds and 0.8 seconds. For example, when the first threshold and the second threshold are 1.2 seconds, the high-quality video reception time is greater than or equal to the first threshold and the super-resolution processing time is less than the second threshold, so the initial video quality determination unit 13 determines the initial video quality to be low-quality video and determines that super-resolution processing will be performed.

[0035] The segment acquisition unit 14 extracts the URL of a segment associated with a playback time and having a video quality selected by the initial video quality determination unit 13 from the segment URL list input from the manifest acquisition unit 11. Then, the segment acquisition unit 14 generates a segment request including the URL and transmits it to the distribution server 3. When the segment acquisition unit 14 obtains the corresponding segment from the distribution server 3 as a response to the request, it accumulates it in the buffer 15.

[0036] The buffer 15 is a memory for temporarily storing the segments input from the segment acquisition unit 14 .

[0037] The decoding unit 16 reads out from the buffer 15 a segment that has been acquired by the segment acquisition unit 14 and that has been requested by the playback unit 18. The decoding unit 16 then decodes the segment read out from the buffer 15 in accordance with the media format (video, audio, text, etc.) to generate decoded data and outputs it to the super-resolution processing unit 17.

[0038] When the super-resolution determination information input from the initial video quality determination unit 13 indicates that super-resolution processing is to be performed, the super-resolution processing unit 17 performs super-resolution processing on the decoded data input from the decoding unit 16. That is, when the high-quality video reception time (first time) is equal to or greater than the first threshold T1 and the super-resolution processing time (third time) is less than the second threshold T2, the super-resolution processing unit 17 performs super-resolution processing. The super-resolution processing unit 17 can use any known super-resolution method such as SRCNN (super-resolution convolutional neural network), which is a learning-type super-resolution method. Note that super-resolution technology has been actively researched and developed in recent years as a technology for beautifully increasing the resolution of digital images, and as described in the following references, there are learning-type super-resolution and reconstruction-type super-resolution. Various learning-type super-resolution methods using deep learning have also been proposed, which ensure the image quality achieved by conventional learning-type super-resolution methods and enable a significant reduction in processing time. [References] "4K / 8K TV and Super-Resolution Technology", Journal of the Institute of Image Information and Television Engineers, Vol. 69, No. 6, 2015, pp. 548-552

[0039] When a playback command including playback start time information is input from the user interface, the playback unit 18 plays back and displays the segment based on the playback start time.

[0040] The above-mentioned reception method functions when video streaming begins, or when playback is temporarily interrupted due to a seek operation or the like and playback is restarted; from that point on, playback is carried out using the conventional adaptive streaming mechanism.

[0041] (Processing at the start of playback) Next, the process performed by the receiving device 1 when playback starts will be described with reference to the flowchart in Fig. 3. The flowchart in Fig. 3 starts from a state in which a manifest file, a video quality list, and a super-resolution processing time list have already been acquired.

[0042] In step S101, the initial video quality determination unit 13 calculates a low-quality video receiving time, which is the time required for buffering low-quality video.

[0043] In step S102, the initial video quality determination unit 13 calculates a super-resolution processing time, which is the sum of the low-quality video reception time and the delay time during the super-resolution processing.

[0044] In step S103, a high-quality video receiving time, which is a time required for buffering high-quality video, is calculated by the initial video quality determining unit 13. Note that the process of step S103 may be performed prior to step S101.

[0045] In step S104, the initial video quality determination unit 13 determines whether the high-quality video reception time is less than the first threshold value. If the high-quality video reception time is less than the first threshold value, the process proceeds to step S105. If the high-quality video reception time is equal to or greater than the first threshold value, the process proceeds to step S106.

[0046] In step S105, the segment acquisition unit 14 acquires a segment of high-quality video.

[0047] In step S106, the segment acquisition unit 14 acquires a segment of the low-quality video.

[0048] In step S107, the high-quality video segment data acquired in step S105 or the low-quality video segment data acquired in step S106 is buffered by the buffer 15. Then, the decoding unit 16 decodes the buffered segment data.

[0049] In step S108, it is determined whether the super-resolution processing time is less than a second threshold value. If the super-resolution processing time is less than the second threshold value, the process proceeds to step S109. If the super-resolution processing time is equal to or greater than the second threshold value, the process of step S109 is not performed.

[0050] In step S109, the super-resolution processor 17 performs super-resolution processing.

[0051] In step S110, the decoded segment data is reproduced.

[0052] In FIG. 3, the case where the video quality of a segment is two types, high quality and low quality, has been described. When video streaming starts or playback is temporarily interrupted due to a seek operation or the like and then resumed, the delay until playback starts can be shortened by performing such a threshold comparison. In addition, if the threshold is not exceeded, super-resolution processing can be performed, so that degradation of image quality at the start of playback can be suppressed. After the start or restart of playback, as in the conventional method, the video quality is selected from the manifest file at the appropriate time taking into account the network bandwidth status of the transmission path, and the segment of the selected video quality is obtained and decoded.

[0053] (program) A computer capable of executing program instructions may be used to function as the above-mentioned receiving device 1. Here, the computer may be a general-purpose computer, a special-purpose computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing the necessary tasks.

[0054] The computer includes a processor, a storage unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be composed of multiple processors of the same type or different types. The processor reads a program from the storage unit and executes it to control each of the above components and perform various arithmetic processing. Note that at least a part of the processing contents may be realized by hardware. The input unit is an input interface that receives a user's input operation and acquires information based on the user's operation, and is a pointing device, a keyboard, a mouse, or the like. The output unit is an output interface that outputs information, and is a display, a speaker, or the like. The communication interface is an interface for communicating with an external device.

[0055] The program may be recorded on a computer-readable recording medium. By using such a recording medium, the program can be installed on the computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. In addition, the program may be in a form that is downloaded from an external device via a network.

[0056] As described above, the receiving device 1 or the program for functioning as the receiving device 1 calculates the first time, the second time, and the third time before playing back video content, and when the first time is less than the first threshold, determines the initial video quality to be the video quality corresponding to the first time, and when the first time is equal to or greater than the first threshold, determines the initial video quality to be the video quality corresponding to the longest time among the second time and the third time that is equal to or less than the second threshold, obtains and decodes a segment of the initial video quality, and when the initial video quality is the video quality corresponding to the third time, further performs super-resolution processing on the decoded data. This makes it possible to reduce the delay until the start of playback in streaming video playback, while suppressing degradation of image quality at the start of playback.

[0057] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple components or process steps described in the embodiment into one, or to divide one into multiple components. [Explanation of symbols]

[0058] 1. Receiving device 2 Content Generation Device 3. Distribution Server 11 Manifest Acquisition Department 12 Super-resolution processing time management section 13 Initial image quality determination unit 14 Segment Acquisition Section 15 Buffer 16 Decoding section 17 Super-resolution processing section 18 Playback Department

Claims

1. A receiving device for receiving a segment selected from a plurality of types of video quality, an initial video quality determination unit that calculates a first time required to receive a segment of the highest video quality, one or more second times required to receive a segment of each of the video qualities other than the highest, and one or more third times required to receive and super-resolution process a segment of each of the video qualities other than the highest before playing back a video content, and determines an initial video quality to be the video quality corresponding to the first time if the first time is less than a first threshold, and determines an initial video quality to be the video quality corresponding to one of the second time and the third time that is less than or equal to the second threshold if the first time is equal to or greater than the first threshold; a segment acquisition unit that acquires a segment having a video quality determined by the initial video quality determination unit; a decoding unit that decodes the segment acquired by the segment acquisition unit to generate decoded data; a super-resolution processing unit that performs a super-resolution process on the decoded data when the initial video quality determination unit determines the video quality corresponding to the third time as the initial video quality; A receiving device comprising:

2. the first time is a time required for buffering high quality video; the second time is a time required for buffering the low quality video; the third time is a sum of a delay time required until playback starts when the low-quality video is subjected to super-resolution processing and the second time, the initial image quality determination unit determines an initial image quality to be the high quality image when the first time is less than the first threshold, and determines an initial image quality to be the low quality image when the first time is equal to or greater than the first threshold; The receiving device according to claim 1 , wherein the super-resolution processing unit performs the super-resolution processing when the first time is equal to or greater than the first threshold value and the third time is less than the second threshold value.

3. a super-resolution processing time management unit that manages a super-resolution processing time list indicating the processing time required for super-resolution processing for each display resolution of the image; The receiving device according to claim 2 , wherein the initial video quality determination unit determines the delay time based on the super-resolution processing time list.

4. a manifest acquisition unit that acquires and analyzes a manifest file, generates a segment URL list in which playback times of segments correspond to URLs of the segments, and generates a video quality list in which bit rates and resolutions of segments of each video quality correspond to each other; The initial video quality determination unit calculates the first time and the second time based on the video quality list; The receiving device according to claim 1 , wherein the segment acquisition unit extracts a URL of a segment having the video quality determined by the initial video quality determination unit from the segment URL list, and generates a segment request including the URL.

5. A program for causing a computer to function as the receiving device according to any one of claims 1 to 4.

Images