Receiving terminal and segment acquisition control method

Abstract

To control video quality on the basis of video scene importance obtained from contents and a shade of meaning, included in the video scene, in streaming video distribution.SOLUTION: A reception terminal for receiving streaming video distribution comprises a video quality determination part that performs reception bit rate control of a segment while performing adjustment within a rate determination period on the basis of a scene list where video scene importance is described in units of the segment obtained by dividing a video stream. This enables a video to be watched with video quality giving consideration to the video scene importance.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a receiving terminal that receives streaming distribution of moving images and a segment acquisition control method in the receiving terminal, and more particularly to a receiving terminal and a segment acquisition control method that control video quality based on the importance of video scenes in streaming distribution of moving images. [Background technology]

[0002] In recent years, the mainstream method for streaming video on the Internet is to use general-purpose web servers to deliver streaming content using the HTTP protocol to many devices. There are a variety of proprietary technologies developed by IT vendors for this type of HTTP streaming delivery method (adaptive streaming), as well as the international standard MPEG-DASH (ISO / IEC23009-1), which aims to unify these streaming methods. The basic concept of both technologies is the same: a web server prepares video streams encoded with one or more video qualities (screen size and bit rate) that are divided into files (segments) of several seconds to several tens of seconds each, along with a manifest file that describes the attributes and URLs of the video content. The receiving terminal selects the appropriate video quality from the manifest file, taking into account the screen size of the receiving terminal and the network bandwidth status of the transmission path, receives the segments one after another, and connects them together to form a single video content for playback (Non-Patent Document 1).

[0003] There are no standards for selecting video quality, and a widely used method is to control it using factors such as the network status and the remaining buffer capacity of the viewing terminal (Non-Patent Document 2).In addition, a method for selecting a bit rate (video quality) according to the level of scene complexity (Patent Document 1) has been disclosed, in which segments are scored and video quality is selected based on this score. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2014-506748 [Non-patent literature]

[0005] [Non-Patent Document 1] Mitsuhiro Hirabayashi, "Next Generation Video Streaming Technology "MPEG-DASH" Technical Overview and Standardization / Related Technology Trends", Journal of the Institute of Image Information and Television Engineers, 2013, Vol. 67, No. 2, pp. 109-115 [Non-patent document 2] Hongzi Mao et al., “Neural Adaptive Video Streaming with Pensieve”, SIGCOMM '17, August 21-25, 2017, pp 197-210 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the scoring of these segments is based on an index of scene complexity (such as the intensity of movement within the scene), and does not take into account the content or meaning contained in the video scene. The present invention has been made in consideration of such problems, and its purpose is to control video quality in streaming video distribution based on the importance of video scenes obtained from the content and meaning contained in the video scenes. [Means for solving the problem]

[0007] In order to solve the above problems, one aspect of the present invention is characterized in that (1) a receiving terminal that receives streaming distribution of video, which is equipped with a video quality determination unit that controls the receiving bit rate of segments while adjusting it within a rate determination period based on a scene list that describes the importance of video scenes in segments into which a video stream is divided, thereby enabling video viewing at a video quality that takes into account the importance of the video scenes.

[0008] Furthermore, a further aspect of the present invention is a receiving terminal (1) characterized in that (2) the receiving bit rate of the segment is controlled based on a scene list describing multiple patterns of importance according to the user of the receiving terminal.

[0009] Another aspect of the present invention is (3) a segment acquisition control method for controlling video quality in streaming video distribution, taking into account the importance of video scenes, which is characterized by controlling the distribution bit rate of segments while adjusting it within a rate determination period based on a scene list that describes the importance of video scenes in segments obtained by dividing a video stream, thereby enabling video viewing at video quality that takes into account the importance of video scenes.

[0010] Furthermore, another aspect of the present invention is (4) a segment acquisition control method (3), characterized in that the delivery bit rate of the segment is controlled based on a scene list describing multiple patterns of importance according to the user watching the video. [Effects of the Invention]

[0011] According to the present invention, it is possible to control video quality based on the importance of video scenes in adaptive streaming. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a content distribution system using a receiving terminal and a segment acquisition method according to the present invention.

[0013] [Figure 2] FIG. 2 is a flowchart showing an example of a method for determining a combination of video qualities for a rate determination period according to the present invention.

[0014] [Figure 3] FIG. 3 is a flowchart showing an example of a segment acquisition control method of the receiving terminal 100 according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] 1 includes a communication interface 101, a manifest acquisition unit 102, a scene list acquisition unit 103, a throughput measurement unit 104, a segment acquisition unit 105, a video quality determination unit 106, a buffer 107, a playback unit 108, and a display unit 109. In this embodiment, the description will be given assuming MPEG-DASH as an example of a content distribution method.

[0017] When a program is selected, the manifest acquisition unit 102 generates a manifest request and transmits it to the content distribution server 200 via the Internet through the communication interface 101. Then, as a response to the manifest request, the manifest acquisition unit 102 acquires a manifest file (MPD file) of the desired program from the content distribution server 200. Then, based on the manifest file, the manifest acquisition unit 102 generates a segment URL list in which segment start times and segment URLs are associated with each other, and outputs the segment URL list to the video quality determination unit 106.

[0018] The scene list acquisition unit 103 generates a scene list request and sends it to the content distribution server 200 via the Internet via the communication interface 101, thereby acquiring a scene list containing importance scores corresponding to the segment numbers of the video stream.

[0019] Here, we will explain how to generate a scene list. The scene list is generated by the side that performs streaming distribution (distribution side). In the present invention, it is preferable that the distribution side describe multiple patterns of importance scores in the scene list. This allows the receiving side (receiving terminal 100) to control the video quality based on the importance of the video scene obtained from the content and meaning contained in the video scene, as will be described later.

[0020] Information about the people appearing in a video scene may be used to score the importance obtained from the content and meaning of the video scene. For example, if the target content has three people A, B, and C appearing, an importance score for each person is calculated for the video scene, resulting in three different importance scores. If the length of one segment (segment length) is one second and the frame rate is 30 fps, the number of frames in one segment is 30. In this case, the number of frames in which the target person appears out of all the frames (30 frames) in one segment may be calculated and used as the importance score for the segment. Note that the segment length is not limited to one second, but the following explanation assumes that the segment length is one second.

[0021] An example of a scene list in this case is shown in Table 1. The importance scores for the segment number are written for each pattern of characters A, B, and C. In the example in Table 1, within segment number 1 (30 frames in total), character A appears in 10 frames, character B appears in 0 frames, and character C appears in 30 frames, so the importance scores for the three patterns for segment number 1 are 10, 0, and 30, respectively.

[0022] [Table 1]

[0023] Although the importance scoring method has been described using person information and the number of frames in which the person appears as an example, object information other than people may also be used, or the degree of screen occupancy may be used instead of the number of frames in which the person appears. Furthermore, scoring may be performed based on the meaning of frames other than the person information or object information appearing, or multiple patterns of importance scores may be set to correspond to patterns such as pre-identified user preferences.

[0024] In any case, in the present invention, the distribution side sets multiple patterns of importance scores based on the content and meaning contained in the video scenes and describes them in a scene list. As will be described in detail later, in the present invention, the receiving terminal 100 controls the video quality of the segments it acquires using an importance pattern extracted from the multiple patterns of importance scores (importance patterns) described in the scene list.

[0025] The segment acquisition unit 105 inputs the URL of the segment to be acquired output by the manifest acquisition unit 102 from the video quality determination unit 106, sends a request including the URL of the input segment to the content distribution server 200 via the Internet via the communication interface 101, receives the segment from the content distribution server 200 as a response to the request, outputs it to the buffer 107, and also outputs information on the time required for the request and the number of bytes of the acquired segment to the throughput measurement unit 104.

[0026] The throughput measurement unit 104 calculates a throughput value from the information on the time required for the request and the number of bytes of the acquired segment input from the segment acquisition unit 105. For example, the throughput value may be calculated as follows: throughput value = (number of bytes of the segment) / (time required for the request).

[0027] At the start of playback or when the final segment of a rate determination cycle is acquired, video quality determination unit 106 calculates the target distribution capacity for the next rate determination cycle to be acquired based on the throughput value input from throughput measurement unit 104, extracts an importance pattern according to the user of receiving terminal 100 input by display unit 109 from the scene list input by scene list acquisition unit 103, and determines the video quality of each segment for the next rate determination cycle to be acquired. The input of importance patterns by display unit 109 will be described later.

[0028] Here, the rate determination cycle can be preset such that one cycle = N segments (e.g., N = 5), and adjustments are made within this cycle so that segments with high video scene importance become high-quality video streams. At this time, an array representing a combination of video qualities (bit rates) that includes as many high-quality segments as possible while staying within the target distribution capacity is derived, and the video quality of each segment for the next rate determination cycle is determined by assigning the highest video quality bit rate from the combination array to the segments in descending order of importance score. A specific method for deriving the array will be described later.

[0029] The remaining buffer capacity (the difference between the final time of the segment inserted in buffer 107 and the current playback time) is calculated from the current playback time input from playback unit 108 and the buffering information input from buffer 107, and if this remaining buffer capacity is smaller than the buffer threshold (for example, 1 second) and the calculated throughput value is smaller than the determined video quality, a video quality that is smaller than the determined video quality and has the maximum bit rate is selected, and the URL of the segment is output to segment acquisition unit 105.

[0030] Buffer 107 receives segments from segment acquisition unit 105, stores the segments by associating their playback start times with their playback duration times (segment lengths), and sequentially outputs segments with playback times requested by playback unit 108 to playback unit 108. Buffer 107 also outputs buffering information, which is information on the playback times and playback durations of all currently stored segments, to video quality determination unit 106.

[0031] The playback unit 108 sequentially inputs segments from the buffer 107 according to the current playback time, decodes the input segments according to the media format of the segments (video, audio, text, etc.), and outputs the decoded segments to the display unit 109 .

[0032] The display unit 109 displays the decoded media data input from the playback unit 108 in accordance with the format of the media. The display unit 109 also accepts an input of an importance pattern by the user, and outputs information about the importance pattern selected by the user to the video quality determination unit 106.

[0033] Here, a method for determining a combination of video qualities for each segment for a rate determination cycle according to the present invention will be explained using the flowchart in Figure 2. Here, the video quality determination unit 106 derives an array representing a combination of video qualities that includes a larger number of high-quality segments that are smaller than the target delivery capacity, which is (throughput value) x (number of segments N for the rate determination cycle) at the time of acquiring the final segment for the rate determination cycle input from the throughput measurement unit 104. There are three types of video quality (bit rate): high quality, medium quality, and low quality. Note that the number of the first segment for the rate determination cycle determined here is designated as X.

[0034] When the video quality combination determination process starts, first in S101 it is determined whether (low-quality bit rate) x (number of segments N for the rate determination cycle) is smaller than the target distribution capacity. If it is not smaller (NO in S101), proceed to S102, and if it is smaller (YES in S101), proceed to S104.

[0035] In S102, N low-quality bit rates are added to the combination array of video quality, and the process proceeds to S103. In S103, the elements of the determined array are assigned in descending order of importance score from segment (X) to segment (X+N), and the process ends.

[0036] In S104, it is determined whether there is remaining capacity to add a high-quality bit rate. If it is possible to add it (YES in S104), the process proceeds to S105, and if it is not possible to add it (NO in S104), the process proceeds to S107.

[0037] In S105, the high quality bit rate is added to the array, and the process proceeds to S106. In S106, it is determined whether all elements of the array have been determined. If all elements of the array have been determined (YES in S106), the process proceeds to S103, and if not all elements of the array have been determined (NO in S106), the process returns to S104.

[0038] In S103, the elements of the determined array are assigned in descending order of importance score from segment (X) to segment (X+N), and the process ends. In S107, it is determined whether there is remaining capacity to add a medium quality bitrate. If it is possible to add it (YES in S107), the process proceeds to S108, and if it is not possible to add it (NO in S107), the process proceeds to S110.

[0039] In S108, the medium quality bitrate is added to the array, and the process proceeds to S109. In S109, it is determined whether all elements of the array have been determined. If all elements of the array have been determined (YES in S109), the process proceeds to S103, and if not all elements of the array have been determined (NO in S109), the process returns to S107.

[0040] In S110, it is determined whether or not there is remaining capacity to add a low-quality bit rate. If it is possible to add it (YES in S110), the process proceeds to S111, and if it is not possible to add it (NO in S110), the process proceeds to S113.

[0041] In S111, the low-quality bitrate is added to the array, and the process proceeds to S112. In S112, it is determined whether all elements of the array have been determined. If all elements of the array have been determined (YES in S112), the process proceeds to S103, and if not all elements of the array have been determined (NO in S112), the process returns to S110.

[0042] In S113, it is determined whether the last element in the array is low quality, and if it is (YES in S113), proceed to S114, and if it is not (NO in S113), skip S114 and proceed to S115.

[0043] In S114, the final element is deleted and the process proceeds to S115. In S115, the quality of the final element is further stored, and then the final element is deleted and the process proceeds to S116.

[0044] In S116, it is determined whether or not there are elements in the array. If there are elements in the array (YES in S116), the process proceeds to S117, and if there are no elements in the array, the process proceeds to S118.

[0045] In S117, it is determined whether the final element is of high quality. If the final element is of high quality (YES in S117), the process proceeds to S107, and if the final element is not of high quality (NO in S117), the process proceeds to S110.

[0046] In addition, in S118, it is determined whether the stored quality is high quality. If the stored quality is high quality (YES in S118), the process proceeds to S107, and if the stored quality is not high quality (NO in S118), the process proceeds to S110.

[0047] The above flow makes it possible to determine a combination sequence of video qualities for the number of segments equal to the rate determination cycle.

[0048] Next, a segment acquisition control method in the receiving terminal 100 of the present invention will be described using the flowchart in Fig. 3. Note that this flowchart starts from a state where a manifest file and a scene list have already been acquired, and an importance pattern appropriate for the user has already been extracted from multiple patterns of importance scores (importance patterns) described in the acquired scene list.

[0049] When playback of a segment starts, first in S201 it is determined whether playback has started or the final segment of the rate determination cycle is currently being acquired. If playback has started or the final segment of the rate determination cycle is currently being acquired (YES in S201), the process proceeds to S202, and if playback has not started or the final segment of the rate determination cycle is not currently being acquired (NO in S201), the process skips S202 and proceeds to S203.

[0050] In S202, the target distribution capacity for the rate determination cycle to be acquired next is calculated, and the video quality of each segment for the rate determination cycle to be acquired next is determined based on the combination array of video qualities (bit rates) derived by the video quality determination unit 106 and the importance pattern according to the user in the scene list input from the scene list acquisition unit 103. When playback starts, the target distribution capacity may be calculated based on the throughput value at the time of the previous playback.

[0051] Next, in S203, the remaining buffer capacity (the difference between the final time of the segment inserted in the buffer and the current playback time) is calculated from the current playback time input from playback unit 108 and the buffering information input from buffer 107, and it is determined whether this remaining buffer capacity is smaller than a buffer threshold (for example, 1 second) and whether the calculated throughput value is smaller than the determined video quality input from video quality determination unit 106. If it is smaller (YES in S203), the process proceeds to S204, and if it is not smaller (NO in S203), the process skips S204 and proceeds to S205.

[0052] In S204, a video quality that is smaller than the determined video quality and has the maximum bit rate is selected.

[0053] Next, in S205, the segment request (including the segment URL of the selected video quality) input from the video quality determination unit 106 is sent to the distribution server 200, segment data in response to the request is received, and the process proceeds to S206.

[0054] In S206, the received segment data is buffered, and the process proceeds to S207. In S207, the segment data is played back, and the process proceeds to S208.

[0055] In S208, it is determined whether the current playback time has reached the end time of the content or whether the stop button has been pressed. If the current playback time has not reached the end time of the content and the stop button has not been pressed (NO in S208), the process returns to S201. If the current playback time has reached the end time of the content or the stop button has been pressed (YES in S208), the process ends.

[0056] As a result, video streaming distribution enables video viewing with video quality that takes into account the importance of video scenes. Furthermore, since the importance derived from the content and meaning contained in a video scene varies depending on the user, by describing multiple patterns of importance scores in the scene list, it becomes possible to control video quality based on the importance of a scene according to the user. [Explanation of symbols]

[0057] 100 receiving terminal 101 Communication Interface 102 Manifest Acquisition Department 103 Scene list acquisition unit 104 Throughput measurement unit 105 Segment Acquisition Unit 106 Video quality determination unit 107 buffers 108 Playback Department 109 Display section 200 Content distribution server

Claims

1. A receiving terminal for receiving streaming video, A receiving terminal characterized by having a scene list that describes the importance of video scenes in units of segments divided from a video stream, and a video quality determination unit that controls the receiving bit rate of segments while adjusting it within a rate determination period based on the scene list that describes multiple patterns of importance corresponding to the user of the receiving terminal, thereby enabling video viewing at a video quality that takes into account the importance of video scenes.

2. A segment acquisition control method for controlling video quality in streaming video distribution, taking into account the importance of video scenes, comprising: A segment acquisition control method characterized by a scene list that describes the importance of video scenes in units of segments divided from a video stream, and by controlling the delivery bit rate of segments while adjusting it within a rate determination period based on the scene list that describes multiple patterns of importance according to the user watching the video, thereby enabling video viewing with video quality that takes into account the importance of the video scenes.

Images