Receiving device and program

Abstract

To reduce a video switching delay without impairing stability.SOLUTION: A reception device 1 includes: a manifest acquisition unit 11 that analyzes a received manifest and generates a segment URL list in which a segment reproduction time and URL are associated for each media identifier; a buffer control unit 14 that extracts from the segment URL list a URL of a segment for which a buffer time that is obtained by adding a predetermined buffer value to a present reproduction time is included in a reproduction time range in a media identifier after switching after the media component is switched if there is, in a buffer 15, a segment for which the buffer time is included in a reproduction time range in receiving a media component switching instruction; and a segment acquisition unit 13 that obtains a segment based on the URL extracted by the buffer control unit 14.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a receiving device and a program. [Background technology]

[0002] In recent years, streaming video distribution on the Internet has become mainstream using a general-purpose web server with the HTTP protocol. One example of such a streaming distribution method using the HTTP protocol (adaptive streaming) is the international standard MPEG-DASH (ISO / IEC23009-1).

[0003] In MPEG-DASH, a web server is provided with initialization segments, media segments, and a manifest file describing their URLs and video content attributes. Media segments are segments obtained by dividing video content streams encoded at one or more qualities (parameters such as screen size and bit rate) into files of several to several tens of seconds each. Initialization segments contain metadata necessary for presenting the segments, such as generation parameters related to the encoding and encryption of each stream. A receiving device selects the appropriate quality from the manifest file, taking into account the screen size of the receiving device and the network bandwidth status of the transmission path, receives the segments one after another, and splices them into a single video content for playback (see, for example, Non-Patent Document 1).

[0004] In the receiving device, by using a web browser, which is becoming increasingly popular among many devices, it is possible to create a viewing environment that is compatible with multiple devices (PCs, television receivers, tablet terminals, etc.). For streaming playback, it is possible to use MSE (Media Source Extensions), standardized by the World Wide Web Consortium (W3C), as an extension function of the HTML video element (video tag) (see, for example, Non-Patent Document 2). By using MSE, the video element can reference a buffer as a source. Received segments can be inserted into this buffer using Javascript (registered trademark), making it possible to view a variety of adaptive streaming using HTML5 video elements and Javascript.

[0005] Here, the MSE buffer model will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the MSE buffer model. The buffer (SourceBuffer) manages segments by associating their playback start time with their playback duration (segment length). In Fig. 6, the shaded blocks represent segments, and three segments have been inserted into the buffer: one with a playback start time of 0 seconds and a playback duration of 5 seconds (0≦playback time t<5), one with a playback start time of 5 seconds and a playback duration of 5 seconds (5≦playback time t<10), and one with a playback start time of 15 seconds and a playback duration of 5 seconds (15≦playback time t<20). In this case, the buffering ranges (buffered attribute) of the buffer are 0 to 10 seconds and 15 to 20 seconds.

[0006] As shown in Figure 7, it is also possible to insert another segment whose playback time t overlaps with a range where a segment has already been inserted, overwriting the existing range. Figure 7 shows an example in which a new segment with a playback start time of 12 seconds and a playback duration of 10 seconds is inserted into the buffer shown in Figure 6. As a result, the range from 12 seconds to 22 seconds is replaced with the new segment, and the buffering range is updated to 0 to 10 seconds and 12 to 22 seconds.

[0007] Next, the manifest file will be described with reference to Fig. 8. Fig. 8 is a conceptual diagram of a manifest file in the MPEG-DASH manifest (MPD: Media Presentation Description) format. One possible method for providing video streaming services is a service format (multi-angle service) in which desired videos are selected from multiple media components and viewed by switching between them, such as making it possible to view videos of the same content from different angles in addition to the main video. MPEG-DASH defines a method for describing manifests that enables such services.

[0008] The MPD format has a hierarchical structure consisting of Period, AdaptationSet, and Representation in that order. A Period indicates information about video content separated in the time direction. The example in Figure 8 shows that Periods, each with a content duration of 300 seconds, are linked together. A Period consists of one or more (three in the example in Figure 8) AdaptationSets.

[0009] An AdaptationSet indicates information about media components (video, audio, text, etc.) and can include multiple components with different expressions in the same media format (video from different angles, audio in different languages, etc.). In the example in Figure 8, there are two video components (mimeType=video / mp4) and one audio component (mimeType=audio / mp4). Information about each media component included in an AdaptationSet is indicated by a Representation. A Representation consists of one or more segments.

[0010] By creating such a manifest file, it is possible to sequentially receive the segments (Initialization Segment, Media Segment in Figure 8) that form the media component (Adaptation Set) selected in the Period, and by reselecting the media component, it is possible to switch to receiving a segment that forms a different media component and display a different video.

[0011] On the other hand, Patent Document 1 discloses a video switching method for streaming video data as a technique for seamlessly switching between representations of multimedia content while streaming using a network streaming protocol. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-209751 [Non-patent literature]

[0013] [Non-Patent Document 1] Mitsuhiro Hirabayashi, "Next Generation Video Streaming Technology 'MPEG-DASH' Technical Overview and Trends in Standardization and Related Technologies," Journal of the Institute of Image Information and Television Engineers, Vol. 67, No. 2, 2013, pp. 109-115 [Non-patent document 2] W3C, “Media Source Extensions W3C Candidate Recommendation 17 November 2016,” [Retrieved January 12, 2024], Internet<URL:http: / / www.w3.org / TR / media-source / > Summary of the Invention [Problem to be solved by the invention]

[0014] When watching streaming content, the receiving bandwidth constantly fluctuates depending on the network congestion. Therefore, to improve viewing stability, it is common to receive data from several seconds to several tens of seconds ahead of the current playback time (hereinafter referred to as "current playback time") and insert it into a buffer. Therefore, when trying to switch between media components that share the same time axis, there is the issue that the response time to the switch is slow and it can take a long time to switch.

[0015] Figure 9 is a diagram illustrating the above problem. In Figure 9, the media component before switching is AdaptationSet#1, the media component after switching is AdaptationSet#2, and the playback duration of the segment is 5 seconds. It is assumed that switching of the media component is triggered by a switching instruction.

[0016] FIG. 9(a) shows the playback state immediately before a media component switching instruction is issued, in which a 10-second segment is acquired and inserted into the buffer ahead of the current playback time.

[0017] 9(b) shows an example in which, after a media component switching instruction is given, a segment of AdaptationSet#2 is received from the playback time following the last segment inserted in the buffer. In this case, the media component switches approximately 10 seconds after the switching instruction, which may result in a slow response time before the switch, which may impair user convenience.

[0018] 9(c) shows an example in which, after a media component switching instruction is issued, a segment of AdaptationSet#2 is received from the playback time following the segment including the current playback time. In this case, if the segment of AdaptationSet#2 is acquired in time, the media component is switched immediately after the media component switching instruction. However, if the network is congested and the playback time of the segment is not met, the segment cannot be overwritten, and playback of the previously acquired AdaptationSet#1 continues. Therefore, not only is the acquired segment of AdaptationSet#2 wasted, but the buffering time of AdaptationSet#1 is reduced, which can result in unstable playback.

[0019] Furthermore, Patent Document 1 does not disclose the detailed state of the buffer, and the technology disclosed in Patent Document 1 cannot solve the above problem.

[0020] The object of the present invention, which has been made in consideration of the above circumstances, is to reduce the delay in switching between videos without compromising stability when streaming content consisting of a main video and videos from different angles (multi-angle) associated with the main video or videos with different playback speeds (fast forward, rewind, etc.). [Means for solving the problem]

[0021] The gist of the present invention for solving the above problems is as follows.

[0022] A receiving device that receives a segment of content and a manifest file composed of a plurality of media components, wherein the manifest file describes a media identifier that is an identifier of each media component and segment information, analyzes the received manifest file, and generates a segment URL list associating the playback time and URL of the segment for each media identifier; a buffer control unit that, when receiving a media component switching instruction, extracts, from the segment URL list, the URL of a segment whose buffer time obtained by adding a predetermined buffer regulation value to the current playback time is within the range of the playback time, for the switched media identifier after the media component is switched; and a segment acquisition unit that acquires a segment based on the URL extracted by the buffer control unit.

[0023] (2) The receiving device according to (1), wherein the buffer control unit extracts, from the segment URL list, the URL of the segment numbered b for the switched media identifier when there is a segment numbered a for the switched media identifier in the buffer whose playback time range includes the buffer time, and the playback start time of the segment numbered b (b < a) for the switched media identifier is greater than the current playback time.

[0024] (3) The receiving device according to (2), wherein when the buffer control unit acquires the segment numbered b for the switched media identifier and the playback start time of the segment is greater than the current playback time, the buffer control unit overwrites and inserts the segment into the buffer.

[0025] (4) A program for causing a computer to function as the receiving device according to any one of (1) to (3).

Advantages of the Invention

[0026] According to the present invention, when content made up of a plurality of media components is streamed, it is possible to reduce the delay in switching between videos without impairing stability. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a diagram showing an outline of a content distribution system including a receiving device according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing an example of the configuration of a receiving device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a diagram showing an example of a manifest file in MPD format received by a receiving device according to an embodiment of the present invention. [Figure 4] 10 is a flowchart illustrating an example of a segment acquisition method when a media component switching operation is performed on a receiving device according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing an example of a buffer state when a media component switching operation is performed on a receiving device according to an embodiment of the present invention. [Figure 6] FIG. 1 is a diagram illustrating an example of a conventional MSE buffer model. [Figure 7] FIG. 1 is a diagram illustrating an example of segment overwriting in a conventional MSE buffer model. [Figure 8] FIG. 1 is a diagram illustrating the concept of a conventional manifest file. [Figure 9] FIG. 1 is a diagram illustrating a conventional problem. DETAILED DESCRIPTION OF THE INVENTION

[0028] An embodiment of the present invention will be described in detail below with reference to the drawings. In the embodiment described below, as an example, the manifest file is in the MPD format of MPEG-DASH.

[0029] 1 shows an overview of a content distribution system including a receiving device according to one embodiment of the present invention. The receiving device 1 is connected to a distribution server 2 via the Internet. The receiving device 1 receives content segments and a manifest file composed of multiple media components of the same media format from the distribution server 2, and performs streaming playback.

[0030] The distribution server 2 is a server that distributes the manifest file and segments of the URL specified by the receiving device 1, and can be, for example, a general web server. The manifest file describes a media identifier that is an identifier for each media component, the playback time and URL of the segment, etc.

[0031] Next, we will explain the details of the receiving device 1. Fig. 2 is a block diagram showing an example configuration of the receiving device 1. The receiving device 1 shown in Fig. 2 includes a communication I / F 10, a manifest acquisition unit 11, a segment list storage unit 12, a segment acquisition unit 13, a buffer control unit 14, a buffer 15, a playback unit 16, and a playback control I / F 17.

[0032] The manifest acquisition unit 11 receives a manifest file of desired content from the distribution server 2 via the communication I / F 10. Then, the manifest acquisition unit 11 analyzes the received manifest file, generates a segment URL list in which the playback time and URL of each segment are associated with each media identifier, and outputs the list to the segment list storage unit 12.

[0033] Figure 3 shows an example (excerpt) of a manifest file in MPD format. In the example shown in Figure 3, there is one Period with a content duration of three minutes, and the Period has two AdaptationSets with the same media format (mimeType="video / mp4") (id="main": main video, id="sub": video from a different angle), and each AdaptationSet has one Representation. In this example, the two AdaptationSets share the same time axis. Hereinafter, the id of the AdaptationSet will be referred to as the "media identifier."

[0034] Next, we will explain how to generate a segment URL list based on the manifest file shown in Figure 3. From the start value on line 2, the start time of the Period is 0. From the timescale value on line 5, the timescale of the Representation (number of divisions per second) is 1. From the timescale and duration values ​​on line 5, the duration of the segment (duration / timescale) is 5 seconds. From the startNumber value on line 5, the start number of the segment is 1. From the above, the segment number Number for a certain playback time t (seconds) can be calculated using the following formula. Note that decimal points are truncated. Number = (t -start) / (duration / timescale) + startNumber

[0035] Next, the segment URL template is "v1_$Number$.mp4" based on the media value on the fifth line, so by replacing $Number$ with the previously calculated Number value, the segment URL for each playback time can be obtained. The second AdaptationSet can be calculated in the same way. In this way, the manifest acquisition unit 11 generates a segment URL list that associates the segment playback time and URL for each media identifier (AdaptationSet id).

[0036] Referring again to Figure 2, segment acquisition unit 13 receives the URL of the segment to be acquired from buffer control unit 14, and transmits a request including the URL of the received segment to distribution server 2 via communication I / F 10. Then, segment acquisition unit 13 receives the segment from distribution server 2 via communication I / F 10 as a response to the request, and outputs it to buffer control unit 14.

[0037] The buffer 15 receives segments from the buffer control unit 14, stores and manages the segments by associating their playback start times and playback durations. To improve viewing stability, the buffer 15 stores data up to a buffer specified value (e.g., 10 seconds) ahead of the current playback time. The buffer 15 then outputs buffered segment range information, which indicates the playback start times and playback durations of all currently stored segments, to the buffer control unit 14. The buffer 15 also sequentially outputs segments with playback times requested by the playback unit 16 to the playback unit 16.

[0038] The playback control I / F 17 is an interface for controlling playback through user operations. The playback control I / F 17 provides buttons for starting and stopping playback. The playback control I / F 17 also obtains information such as the number of media components from the segment URL list stored in the segment list storage unit 12, and provides buttons for switching between media components.

[0039] The playback control I / F 17 may also graphically display the current playback position (current playback time) based on the start and end times of the content and playback position information input from the playback unit 16. For example, the playback control I / F 17 may display the content start time at the left end and the end time at the right end as a horizontally long bar at the bottom of the playback screen, with a knob-like operating part superimposed on the bar. By moving the operating part on the knob from the left end to the right end as playback progresses, the user can grasp at a glance which part of the entire playback time is currently being played. The playback control I / F 17 also outputs instructions such as playback start and stop in response to user operations to the playback unit 16, and outputs instructions to switch media components to the buffer control unit 14.

[0040] When the playback unit 16 receives a playback start instruction from the playback control I / F 17, it sequentially inputs segments from the buffer 15, decodes the input segments according to their media format (video, audio, text, etc.), and displays them on the display. The playback unit 16 also outputs playback position information indicating the current playback position (current playback time) to the buffer control unit 14 and the playback control I / F 17.

[0041] The buffer control unit 14 determines the next segment to acquire based on the media component switching instruction input from the playback control I / F 17, the playback position information input from the playback unit 16, the buffered segment range information input from the buffer 15, and a buffer specified value that is set in advance in anticipation of the fact that the reception bandwidth will constantly fluctuate depending on the network congestion status.The buffer control unit 14 then outputs the URL of the next segment to acquire to the segment acquisition unit 13.The buffer control unit 14 also inputs the segment from the segment acquisition unit 13 and outputs it to the buffer 15.

[0042] <Method for controlling segment acquisition when switching media components> Next, a segment acquisition method when switching media components in the receiving device 1 is described using the flowchart in FIG. 4 and the diagram showing the buffer state in FIG. 5. The flowchart in FIG. 4 assumes that the manifest file has already been acquired by the manifest acquisition unit 11, the segment URL list has already been created, and playback of one of the media components is continuing. Furthermore, in consideration of playback stability, a state in which segments equal to or greater than the buffer specified value are inserted in the buffer 15 is maintained. As will be described in detail below, the segment acquisition method when switching media components includes the steps of: the receiving device 1 analyzing the received manifest file and generating a segment URL list in which the playback time and URL of each segment are associated with each media identifier; upon receiving a media component switching instruction, if a segment exists in the buffer whose playback time range includes a buffer time obtained by adding a specified buffer specified value to the current playback time; extracting from the segment URL list the URL of a segment whose buffer time is within the playback time range for the post-switching media identifier after switching the media component; and acquiring the segment based on the extracted URL.

[0043] In step S001, a media component switching operation is executed in the playback control I / F 17 by a user operation.

[0044] In step S002, when the buffer control unit 14 receives a media component switching instruction from the playback control I / F 17, it extracts the media identifier after switching of the media component. In the following explanation, the time obtained by adding a predetermined buffer specified value to the current playback time is referred to as the "buffer time." As time passes, the current playback time and the buffer time increase.

[0045] In step S003, the buffer control unit 14 checks, based on the buffered segment range information, whether or not a segment whose buffering time falls within the playback time range exists in the buffer 15. If the segment exists (step S003: YES), the process proceeds to step S004; if the segment does not exist (step S003: NO), the process proceeds to step S012. Figure 5(a) shows an example in which the segment exists, and Figure 5(b) shows an example in which the segment does not exist.

[0046] In step S004, the buffer control unit 14 extracts from the segment URL list the URL of a segment (segment number is n) whose buffer time falls within the range of the playback time in the media identifier after switching of the media component. The segment acquisition unit 13 transmits a segment request including the segment URL extracted by the buffer control unit 14 to the distribution server 2, and acquires the segment with segment number n in the media identifier after switching of the media component from the distribution server 2. The buffer control unit 14 overwrites and inserts the acquired segment into the buffer 15. Figure 5(c) shows an example in which the segment with segment number n after switching of the media component is overwritten and inserted into the buffer 15 shown in Figure 5(a).

[0047] In step S005, the buffer control unit 14 sets the counter i=1.

[0048] In step S006, the buffer control unit 14 checks whether the following conditions are met: a segment (segment number is n) whose buffer time is within the playback time range exists in the buffer 15, and the playback start time of a segment (segment number is ni) whose segment number is smaller than that of the segment (segment number is n) is greater than the current playback time. If the conditions are met (step S006: YES), the process proceeds to step S007; if the conditions are not met (step S006: NO), the process proceeds to step S012.

[0049] In step S007, the buffer control unit 14 extracts the URL of the segment with segment number (ni) in the media identifier after switching of the media component from the segment URL list. The segment acquisition unit 13 transmits a segment request including the segment URL extracted by the buffer control unit 14 to the distribution server 2, and acquires the segment with segment number (ni) in the media identifier after switching of the media component from the distribution server 2.

[0050] In step S008, the buffer control unit 14 again checks whether the condition that the playback start time of the segment with segment number (ni) is later than the current playback time is met. If the condition is met (step S008: YES), the process proceeds to step S009. If the condition is not met (step S008: NO), the process proceeds to step S011.

[0051] In step S009, the buffer control unit 14 overwrites and inserts the segment with the segment number (ni) acquired in step S007 into the buffer 15. Fig. 5(d) shows an example in which the segment with the segment number (ni) after switching of the media component is overwritten and inserted into the buffer 15 shown in Fig. 5(c).

[0052] In step S010, the buffer control unit 14 increments the counter i, and returns the process to step S006.

[0053] In step S011, the buffer control unit 14 discards the segment with the segment number (ni) acquired in step S007. Fig. 5(e) shows an example in which the segment with the segment number (ni) after switching of the media component is discarded without being overwritten and inserted into the buffer 15 shown in Fig. 5(c).

[0054] In step S012, the URL of the segment following the buffered segment range in the media identifier after switching of the media component is extracted from the segment URL list. The segment acquisition unit 13 sends a segment request including the segment URL extracted by the buffer control unit 14 to the distribution server 2, and acquires the segment after switching of the media component from the distribution server 2. The buffer control unit 14 inserts the acquired segment into the buffer 15. Thereafter, segments are acquired sequentially using a general segment acquisition method.

[0055] As a result, when streaming content consisting of a main video and multiple sub-videos (content consisting of multiple media components), it is possible to reduce the delay in switching between videos without sacrificing stability. In other words, it is possible to reduce delays and interruptions caused by switching between media components, improving user convenience.

[0056] In this embodiment, the time lengths of the segments of multiple media components are the same, but they do not have to be the same. Also, for each media component, a separate AdaptationSet for media component switching, consisting of segments with the same content but shorter time lengths, may be prepared. By using the segment URL list of the media identifiers in the AdaptationSet only during the period in which the segment acquisition method described above is used, it is possible to reduce the possibility of the acquired segments being discarded and further reduce delays.

[0057] <Program> A computer capable of executing program instructions can also be used to function as the above-described receiving device 1. Here, the computer may be a general-purpose computer, a special-purpose computer, a workstation, a PC, a mobile terminal, etc. The program instructions may be program code, code segments, etc. for performing the necessary tasks.

[0058] The computer includes a processor, a storage unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an SoC (System on a Chip), or the like, and may be configured with multiple processors of the same or different types. The processor reads and executes programs from the storage unit to control the above components and perform various arithmetic processing. Note that at least a portion of these processing contents may be implemented by hardware. The input unit is an input interface that accepts user input operations and acquires information based on the user operations, such as a pointing device, keyboard, or microphone. The output unit is an output interface that outputs information, such as a display or speaker. The communication interface is an interface for communicating with external devices.

[0059] The program may be recorded on a computer-readable recording medium. Using such a recording medium, the program can be installed on a computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0060] The above-described receiving device 1 may be configured with one or more semiconductor chips. The semiconductor chip may be equipped with a CPU that executes a program that describes the processing content for realizing each function of the receiving device 1.

[0061] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications or alterations are possible without departing from the scope of the claims. For example, with regard to the constituent blocks or processing steps described in the embodiments, multiple blocks or processing steps can be combined into one, or one block or processing step can be divided into multiple blocks. [Explanation of symbols]

[0062] 1. Receiving device 2. Distribution Server 10 Communication I / F 11 Manifest Acquisition Department 12 Segment list storage section 13 Segment Acquisition Unit 14 Buffer control section 15 buffers 16 Playback Department 17 Playback control I / F

Claims

1. A receiving device that receives segments of content consisting of multiple media components and a manifest file, The manifest file describes a media identifier, which is an identifier of each media component, and segment information, a manifest acquisition unit that analyzes the received manifest file and generates a segment URL list in which playback times and URLs of the segments are associated with each media identifier; a buffer control unit that, upon receiving a media component switching instruction, extracts, from the segment URL list, a URL of a segment whose playback time range includes a buffer time obtained by adding a predetermined buffer specified value to the current playback time, in a post-switching media identifier after switching of the media component, when a segment whose playback time range includes the buffer time exists in the buffer; a segment acquisition unit that acquires a segment based on the URL extracted by the buffer control unit; A receiving device comprising:

2. The receiving device of claim 1, wherein the buffer control unit extracts the URL of the segment with segment number b in the post-switching media identifier from the segment URL list when there is a segment with segment number a in the post-switching media identifier in the buffer whose playback time range includes the buffer time, and the playback start time of the segment with segment number b (b < a) in the post-switching media identifier is greater than the current playback time.

3. The receiving device described in claim 2, wherein when the buffer control unit acquires the segment with segment number b in the post-switching media identifier, if the playback start time of the segment is greater than the current playback time, the buffer control unit overwrites and inserts the segment into the buffer.

4. A program for causing a computer to function as the receiving device according to claim 1.

Images