Transmitting device, receiving device, and their programs

Abstract

To provide a transmission device capable of reducing video transmission delay.SOLUTION: A transmission device 1 includes a video encoding unit 11 that applies bit rate control to a video signal using a virtual CPB, a chunk configuration unit 12 that configures a chunk from the video signal output at a bit rate equal to or higher than a CBR, a control information generation unit 13 that generates predetermined control information, a transport multiplexer 14 that multiplexes the chunk and the control information into packets, and a transport transmitter 17 that transmits priority data, chunk, and non-priority data as a multiplexed signal in order of priority.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a transmitting device, a receiving device, and programs therefor. [Background technology]

[0002] In a transmission system that handles multiplexed signals including multiple components such as video signals, audio signals, and data, such as broadcast services, operations are generally carried out by allocating a certain range of transmission capacity to each component within the transmission capacity of the transmission path (Non-Patent Document 1). As shown in Fig. 13, a conventional transmission system transmits coded data of a video signal (encoded video signal) at a constant bit rate (CBR: Constant Bit Rate) with the bandwidth allocated to the video signal as the upper limit. Furthermore, conventional transmission systems use, for example, the MMT (MPEG Media Transport) method (Non-Patent Document 2) or the DASH / ROUTE (Dynamic adaptive streaming over HTTP / Real-Time Object Delivery over Unidirectional Transport) method (Non-Patent Document 3) as a multiplexing method for multiple components that make up a broadcast service.

[0003] FIG. 14 illustrates an example of a reference structure between pictures. Here, "I" represents an I (Intra) picture that performs intra-frame compression, "P" represents a P (Predictive) picture that references the preceding frame, and "B" represents a B (Bidirectionally Predictive) picture that references the preceding and succeeding frames. The arrows in FIG. 14 indicate that the picture at the beginning refers to the picture at the end. In actual video coding processing, the amount of coded data of a video signal generated on the time axis varies due to differences in picture type and bit rate control using adjustments such as quantization parameters. Typically, an I picture has the largest amount of coded data, a P picture has the second largest amount of coded data, and a B picture has the smallest amount of coded data (see FIG. 16(a)).

[0004] As shown in FIG. 15, a video encoding unit 9 included in a conventional transmitting device includes an encoding processing unit 90 that encodes a video signal, a coded picture buffer (CPB) 91, and an encoding control unit 92 that controls the bit rate according to the buffer usage of the CPB 91. Furthermore, the video encoding unit 9 is configured with coding parameters of CBR and maximum CPB delay. As shown in FIGS. 16(a) to 16(c), the video encoding unit 9 smooths the coded data in the time direction within the range of the maximum CPB delay so that the bit rate of the coded data is always equal to or less than the CBR, and transmits the smoothed coded data. Specifically, based on the buffer usage fed back from the CPB 91, the encoding control unit 92 determines whether the CPB 91 is about to overflow (whether the coded data cannot be transmitted at CBR and the maximum CPB delay will be exceeded). If the CPB 91 is about to overflow, the encoding control unit 92 increases the compression rate of the video signal to reduce the image quality (reducing the bit amount). On the other hand, if the CPB 91 has room (enough encoded data can be transmitted at CBR and within the maximum CBP delay), the encoding control unit 92 improves the image quality by lowering the compression rate of the video signal (increasing the bit amount). In this way, the encoding control unit 92 controls the output of the CPB 91 so that it does not exceed a predetermined bit rate.

[0005] 16(a) shows the amount of coded data output by the coding processing unit 90. Also, FIG. 16(b) shows the amount of coded data smoothed by CBR, which is output by the CPB 91. Also, FIG. 16(c) shows the amount of coded data output by the transmitting device.

[0006] In recent years, the adoption of the Common Media Application Format (CMAF) for video distribution over the Internet has progressed, unifying the media container formats previously differing depending on the distribution technology, such as MPEG (Moving Picture Experts Group)-DASH (Dynamic Adaptive Streaming over HTTP) and HLS (HTTP Live Streaming) (see Non-Patent Document 4). CMAF standardizes media container formats based on the ISO Base Media File Format (ISOBMFF) and specifies a chunk structure that enables ultra-low latency (ULL) distribution, a technology that enables low-latency distribution. In video distribution using ISOBMFF, on which CMAF is based, without using a chunk structure, it is common to transmit segments with a duration of several to several tens of seconds, starting with a video I-picture. On the other hand, CMAF-ULL allows for the transmission of chunk structures consisting of several frames, which do not necessarily start with a video I-picture. The adoption of CMAF in broadcasting in the future will enable content distribution in a common format across the boundaries between broadcasting and communications, and is expected to lead to the advancement of services that integrate broadcasting and communications.

[0007] When an MPEG-DASH or HLS player (receiving device) starts receiving a service, it first requests and obtains from the server a manifest file that lists segment location information (such as URLs on the server that deliver the segments). Then, according to the segment location information described in the manifest file, it requests and obtains the segments in chronological order from the server and plays back the video. If segment acquisition fails or acquisition takes too long due to poor line conditions, for example, the likelihood of video playback failure due to buffer underflow increases. To avoid such video playback failures, if a segment acquisition failure occurs, the player adjusts by increasing the buffer time within the player to ensure more stable playback.

[0008] In video-on-demand (VOD) content, which viewers can view at any time, all of the segments that make up the content are pre-encoded, and the manifest file can contain the location information for all segments in advance, so the manifest file is not updated during the content's transmission. On the other hand, in live streaming content, which viewers watch live, the manifest file is updated by deleting the old segment information and adding the new segment information each time a new segment is generated by the live encoder. Note that, if catch-up viewing of a live broadcast is also possible, the old segment information may not be deleted, but new segment information may be added and the pointer information indicating the start segment for playback may only be updated. The player periodically retrieves the manifest file and plays the newly added segment. In the case of live streaming, if the player fails to acquire a segment or takes a long time to complete the acquisition, the buffer length is increased to prevent video breakdown. In other words, because more segments are buffered inside the player, the total transmission delay from the time the video captured by the camera is displayed on the player screen increases during live streaming.

[0009] In CMAF-ULL, by using chunks, which have a shorter time scale than segments, the file construction delay, which is the unit of request and acquisition from the server, is reduced, and if the communication conditions are good, it becomes easier to achieve lower latency distribution. However, if chunk acquisition fails or takes a long time to complete, the player increases the buffer size, which increases the total transmission delay, the same principle as in segment-based distribution. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] ARIB TR-B39, "Advanced Wideband Digital Satellite Broadcasting Operational Standards (Volume 2)" [Non-patent document 2] ISO / IEC 23008-1, “High efficiency coding and media delivery in heterogeneous environments: MPEG media transport” [Non-patent document 3] ATSC Standard: A / 331, “Signaling, Delivery, Synchronization, and Error Protection” [Non-patent document 4] ISO / IEC 23000-19, “Common media application format (CMAF) for segmented media” [Non-Patent Document 5] ARIB STD-B60, "Media Transport Method Using MMT in Digital Broadcasting" Summary of the Invention [Problem to be solved by the invention]

[0011] As shown in Figures 15 and 16, in a conventional transmitting device, the output of the video encoding unit 9 passes through a CPB 91, and then metadata is generated for each chunk. The metadata is then transmitted as header information before the chunk is transmitted, resulting in a chunk construction process latency. Therefore, within the transmitting device, a chunk construction latency is added to the video CPB latency (a latency for smoothing the amount of coded data that fluctuates over time). Furthermore, in a conventional receiving device, if a chunk cannot be acquired (is incomplete) when the video decoding unit attempts to acquire the chunk, the buffer is increased to avoid video playback failure. Therefore, the buffering process within the video decoding unit until the video is presented on the screen increases the total transmission latency. The reason why a chunk cannot be acquired is that it takes time to send the chunk completely, since the chunk is transmitted within the transmission capacity allocated to the video component in the broadcast wave. Therefore, the video decoding unit may attempt to acquire the chunk before the entire chunk has been received.

[0012] Therefore, an object of the present invention is to provide a transmitting device, a receiving device, and programs therefor that can reduce video transmission delay. [Means for solving the problem]

[0013] In order to solve the above problem, the transmitting device of the present invention is a transmitting device that receives a video signal, encodes the input video signal, and transmits it, and is configured to include a video encoding unit, a control information generating unit, a chunk construction unit, a multiplexing unit, and a transmitting unit.

[0014] According to this configuration, the video encoding unit applies bit rate control to the video signal using a virtual CPB and outputs the encoded video signal at a bit rate equal to or higher than a preset CBR without going through a CPB. In other words, the video encoding unit virtually emulates a CPB and performs bit rate control similar to that of conventional technology, but since the actual output does not go through a CPB, it can output the video signal at the maximum rate allowed by an internal interface such as a bus. In this way, since the video encoding unit does not go through a CPB, CPB delay can be reduced.

[0015] The chunk construction unit constructs chunks from the video signal output by the video encoding unit at a bit rate of CBR or higher, and adds metadata to the constructed chunks as header information. The metadata includes a list of the data sizes and timestamps of all the video frames that make up the chunk, so the metadata is completed when the encoding of the entire chunk is complete. Once the metadata is complete, the chunk with the metadata attached can be output to the multiplexing unit.

[0016] The control information generator generates control information in which the CBR is described. The multiplexing unit multiplexes the chunks with the metadata and the control information into transmission packets. The transmitter transmits prioritized data including control information with a higher priority than the chunks, the chunks, and non-priority data with a lower priority than the chunks as a multiplexed signal in order of priority so that the transmission rate is equal to or lower than a preset rate. This allows the transmitter to transmit chunks in a shorter time by utilizing the entire bandwidth of the transmission path compared to conventional techniques.

[0017] In addition, in order to solve the above problem, the receiving device of the present invention is a receiving device that receives a multiplexed signal from the transmitting device, and is configured to include a multiplexing separation unit, a manifest file generation unit, a memory unit, a reading unit, and a video decoding unit.

[0018] According to this configuration, the demultiplexing unit separates chunks made up of predetermined control information and encoded video signals from the multiplexed signal. The manifest file generation unit generates a manifest file in which location information of chunks is described according to the control information separated by the demultiplexing unit, and also updates the manifest file immediately when the receiving device completes receiving a new chunk.

[0019] The storage unit stores the chunks and the manifest file. The reading unit reads the chunks and the manifest file from the storage unit. The video decoding unit requests the manifest file from the reading unit, acquires the manifest file from the reading unit in response to the request, and requests and receives chunks indicated by the location information of the acquired manifest file in chronological order to decode the video signal.

[0020] In this way, the transmitting device creates chunks with metadata added in the shortest time possible without using CBR and transmits them in a short time using the entire bandwidth of the transmission path, allowing the chunks to be quickly stored in the memory of the receiving device, thereby quickly responding to acquisition requests from the video decoding unit. Furthermore, the receiving device immediately stores the chunks in the memory upon receiving them and updates the manifest file as soon as the chunks become available for acquisition, making the chunks available for transmission in response to a request from the video decoding unit. This reduces the possibility that the video decoding unit will fail to acquire chunks that have not yet arrived at the receiving device, and prevents unnecessary increases in buffer time to avoid video breakdowns, thereby reducing the total transmission delay of the video.

[0021] The present invention can also be realized by a program for causing a computer to function as the above-mentioned transmitting device or receiving device. [Effects of the Invention]

[0022] According to the present invention, it is possible to reduce video transmission delays. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a block diagram showing a configuration of a transmission device according to an embodiment. [Figure 2] 2 is a block diagram showing the configuration of a video encoding unit in FIG. 1. FIG. [Figure 3] FIG. 10 is an explanatory diagram illustrating transmission in order of priority in an embodiment. [Figure 4] 2 is a flowchart showing the processing of the token output unit of FIG. 1; [Figure 5] 2 is a flowchart showing an interrupt process of the transport transmitter of FIG. 1; [Figure 6] 10 is a flowchart showing a first example of a packet transmission process in the transport transmitter of FIG. [Figure 7] 10 is a flowchart showing a second example of a packet transmission process in the transport transmitter of FIG. [Figure 8] 10A and 10B are explanatory diagrams illustrating the amount of coded data when a multiplexed signal is transmitted in order of priority in an embodiment. [Figure 9] 10A and 10B are explanatory diagrams illustrating the amount of coded data when the transmission of the next and subsequent chunks is delayed in an embodiment. [Figure 10] FIG. 1 is a block diagram showing a configuration of a receiving device according to an embodiment. [Figure 11] FIG. 11 is a sequence diagram showing the operation of the receiving device of FIG. [Figure 12] FIG. 1 is a block diagram showing a configuration of a receiving device according to a reference example. [Figure 13] FIG. 1 is an explanatory diagram illustrating CBR in the prior art. [Figure 14] FIG. 1 is an explanatory diagram illustrating types of pictures in the prior art. [Figure 15] FIG. 1 is a block diagram showing the configuration of a video encoding unit in the prior art. [Figure 16] 1A to 1C are explanatory diagrams illustrating the amount of coded data in a transmitting device according to the prior art. DETAILED DESCRIPTION OF THE INVENTION

[0024] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are intended to embody the technical concept of the present invention, and unless otherwise specified, the present invention is not limited to the following. Furthermore, the same means will be given the same reference numerals, and their description may be omitted.

[0025] [Transmitter configuration] The configuration of a transmission device 1 according to an embodiment will be described with reference to FIG. 1, the transmitting device 1 receives a video signal, encodes the video signal, and transmits the encoded video signal. As shown in Fig. 1, the transmitting device 1 includes an audio encoding unit 10, a video encoding unit 11, a chunk configuration unit 12, a control information generation unit 13, a transport multiplexing unit (multiplexing unit) 14, a chunk queue 15A, a priority data queue 15B, a non-priority data queue 15C, a token output unit 16, and a transport transmitting unit (transmitting unit) 17.

[0026] The audio encoding unit 10 receives an audio signal and encodes the input audio signal. For example, the audio encoding unit 10 encodes the audio signal using a common audio encoding method such as AAC (Advanced Audio Coding), MPEG-H 3D Audio, or AC-4. The audio encoding unit 10 then outputs the encoded audio signal to the chunk configuration unit 12.

[0027] The video encoding unit 11 performs bit rate control using a virtual CPB on a video signal and outputs the encoded video signal at a bit rate equal to or greater than a preset CBR without using a CPB. Here, the video signal is input to the video encoding unit 11, and the maximum CPB delay and the CBR bit rate value are also input as parameters.

[0028] <Video Encoding Section> The video encoding unit 11 will be described in detail with reference to FIG. As shown in FIG. 2, the video encoding unit 11 includes an encoding processing unit 11A, a virtual CPB 11B, and an encoding control unit 11C.

[0029] The encoding processing unit 11A encodes the video signal in accordance with bit rate control from the encoding control unit 11C, which will be described later. For example, the encoding processing unit 11A encodes the video signal using a common video encoding method such as H.265 / HEVC (High Efficiency Video Coding), H.266 / VVC (Versatile Video Coding), or AVI (Audio Video Interleave). The encoding processing unit 11A then inputs the encoded video signal to the virtual CPB 11B and outputs the video signal to the chunk configuration unit 12 without passing through a CPB.

[0030] The virtual CPB 11B is a virtual CPB for emulating the conventional CPB 91 (FIG. 15). In other words, unlike the conventional CPB 91, the virtual CPB 11B does not have a function of actually smoothing and outputting encoded data of a video signal. The virtual CPB 11B is referenced by the encoding control unit 11C.

[0031] The encoding control unit 11C refers to the buffer usage amount of the virtual CPB 11B and performs bit rate control such as selecting an encoding mode in the encoding processing unit 11A and adjusting a quantization parameter. Note that the bit rate control itself of the encoding control unit 11C is similar to that of the conventional encoding control unit 92 (FIG. 15), and therefore a description thereof will be omitted.

[0032] As shown in Figure 2, in the video encoding unit 11, as in the conventional technology, the virtual CPB 11B feeds back the buffer status to the encoding control unit 11C, and the encoding control unit 11C controls the amount of output bits generated by the encoding processing unit 11A. Meanwhile, the video encoding unit 11 outputs coded data of the video signal to the chunk configuration unit 12 without going through the conventional CPB 91 (Figure 15). Therefore, while the output bit rate α (Figure 15) of the conventional video encoding unit 9 is limited by CBR, the output bit rate β (Figure 2) of the video encoding unit 11 is not limited by CBR (β > α). As a result, the chunk configuration unit 12 can output coded data of the video signal without delay at the maximum rate (β) of the internal interface.

[0033] Returning to FIG. 1, the description of the configuration of the transmitting device 1 will continue. The chunk constructing unit 12 constructs chunks with metadata added from the video signal output by the video encoding unit 11 at a bit rate equal to or higher than CBR. In this embodiment, the chunk constructing unit 12 constructs chunks from the video signal output by the video encoding unit 11 at a bit rate equal to or higher than CBR and the audio signal encoded by the audio encoding unit 10. Specifically, the chunk constructing unit 12 constructs a set of frames as a chunk for the video signal. Furthermore, the chunk constructing unit 12 constructs an access unit of the same duration as the set of frames as a chunk for the audio signal. For example, CMAF can be used as the media container format for the chunks. The chunk constructing unit 12 then outputs the chunks constructed from the video signal and the audio signal and with metadata added as header information to the transport multiplexing unit 14.

[0034] The control information generator 13 generates control information in which a CBR is described. For example, when MMT is used as the multiplexing method, the control information generator 13 describes the bit rate value of CBR input as a parameter in a descriptor of an asset loop of an MPT (MMT Package Table) to generate control information. When DASH / ROUTE is used as the multiplexing method, the control information generator 13 describes the bit rate value of CBR input as a parameter in an MPD (Media Presentation Description) to generate control information. Then, the control information generator 13 outputs the generated control information to the transport multiplexing unit 14.

[0035] The transport multiplexing unit 14 multiplexes chunks and control information into transmission packets. Here, the transport multiplexing unit 14 receives a maximum packet length as a parameter and fragments each piece of data according to the received maximum packet length. Specifically, the transport multiplexing unit 14 multiplexes chunks generated by the chunk generating unit 12 into transmission packets and stores the resulting packets in the chunk queue 15A. The transport multiplexing unit 14 also multiplexes control information generated by the control information generating unit 13 into transmission packets and stores the resulting packets in the priority data queue 15B as priority data with a higher priority than chunks. The transport multiplexing unit 14 also receives priority data such as emergency alerts and event messages, multiplexes the input priority data into transmission packets, and stores the resulting packets in the priority data queue 15B. The transport multiplexing unit 14 also receives static data content such as data broadcasting, multiplexes the input static data content into transmission packets, and stores the resulting packets in the non-priority data queue 15C as non-priority data with a lower priority than chunks.

[0036] The chunk queue 15A is for queuing chunks. The priority data queue 15B is for queuing priority data such as control information, emergency alerts, and event messages. The non-priority data queue 15C is for queuing non-priority data such as static data contents.

[0037] The token output unit 16 outputs tokens corresponding to a preset token quantity at predetermined intervals. These tokens are signals used by the transport transmission unit 17 to perform bit rate control. In this embodiment, the token output unit 16 receives the maximum packet length and transmission line capacity as parameters, and calculates the token quantity and token output interval according to the input maximum packet length and transmission line capacity. The token output unit 16 then writes the token quantity in a token and outputs the token to the transport transmission unit 17 at the token output interval. Details of the token output unit 16 will be described later.

[0038] The transport transmitter 17 transmits the priority data, chunks, and non-priority data as a multiplexed signal in order of priority so that the transmission rate is equal to or less than a preset transmission rate. As shown in Fig. 3, the transport transmitter 17 transmits the priority data 150B queued in the priority data queue 15B, the chunks 150A queued in the chunk queue 15A, and the non-priority data queued in the non-priority data queue 15C in order of priority, in response to the token input from the token output unit 16. This ensures that the output bit rate of the transport transmitter 17 is equal to or less than the transmission capacity of the transmission path. The details of the transport transmitter 17 will be described later.

[0039] <Token output processing> The processing of the token output unit 16 will be described in detail with reference to FIG. 4, in step S10, the token output unit 16 calculates the token amount and the token output interval according to the maximum packet length and the transmission path capacity. Specifically, the token output unit 16 can calculate the amount of tokens that can be issued per unit time (1 second) by dividing the transmission path capacity by the maximum packet length. In this case, the token output unit 16 may calculate the amount of tokens per unit time by multiplying the token issuance period, which is the reciprocal of the token output interval, by the token amount per output.

[0040] For example, let's say the transmission channel capacity is 100Mbits per second, the maximum packet length is 1400 bytes, and the number of tokens required to send one packet of the maximum packet length (number of tokens per transmission) is 1.0. In this case, the number of tokens that can be issued per unit time (1 second) is 100,000,000 / (1400*8) = 8929. Therefore, the token output interval is the reciprocal of the number of tokens that can be issued per unit time, or 1 / 8929 seconds.

[0041] If the amount of tokens per unit time remains the same, the amount of tokens per time can be reduced to shorten the issuance interval. For example, if the amount of tokens per time is set to 0.5, the token output interval will be 1 / 17856 seconds. Furthermore, if the amount of tokens per time remains the same, the amount of tokens per time can be increased to lengthen the issuance interval. For example, if the amount of tokens per time is set to 2.0, the token output interval will be 1 / 4464 seconds.

[0042] In step S11, the token output unit 16 outputs to the transport transmission unit 17 the number of tokens corresponding to the amount of tokens calculated in step S10. In step S12, the token output unit 16 waits for the token output interval calculated in step S10, and then returns to the processing of step S11. In other words, the token output unit 16 waits until the next token output timing.

[0043] <Transport Transmitter Processing: Token Interrupt Processing> Referring to FIG. 5, an interrupt process when a token is input to the transport transmitter 17 will be described. As shown in FIG. 5, in step S20, the transport sending unit 17 sets the remaining token amount to zero. In step S21, the transport transmission unit 17 determines whether the remaining token amount is equal to or greater than a preset threshold value (for example, 1).

[0044] If the remaining token amount is equal to or greater than 1 (Yes in step S21), the transport transmission unit 17 proceeds to the process of step S22. In step S22, the transport transmitting unit 17 generates an interrupt to the packet transmitting process of Fig. 6 or Fig. 7. Note that whether the packet transmitting process of Fig. 6 or Fig. 7 is to be executed is manually set by, for example, an administrator of the transmitting device 1. In step S23, the transport sending unit 17 discards the remaining token amount that exceeds the preset upper limit for holding, and returns to the processing of step S21.

[0045] The upper limit of the remaining token amount is a parameter that affects the accuracy of smoothing the output bit rate of the transport transmitter 17, and can be set arbitrarily. The smaller this upper limit is set (minimum value is 0), the more controllable it is to prevent the instantaneous output bit rate of the transport transmitter 17 from exceeding the upper limit bit rate (transmission path transmission capacity). On the other hand, the smaller this upper limit is set, the more likely it is that remaining tokens will be discarded when the number of packets waiting to be transmitted in the queue decreases, resulting in an open space in the transmission bandwidth (a state in which there are no valid packets). On the other hand, the larger this upper limit is set, the less likely it is that remaining tokens will be discarded even when the number of packets waiting to be transmitted in the queue temporarily decreases, resulting in the transmission bandwidth being used without an open space. On the other hand, the larger this upper limit is set, the more likely it is that the instantaneous output bit rate of the transport transmitter 17 will exceed the upper limit bit rate. In other words, there is a trade-off between the efficiency of transmission bandwidth utilization and the accuracy of smoothing the output bit rate of the transport transmitter 17. If the smoothing accuracy is low, separate shaping may be performed in a subsequent stage of the transmitter 1.

[0046] If the remaining token amount is not equal to or greater than 1 (No in step S21), the transport transmission unit 17 proceeds to the process of step S24. In step S24, the transport sending unit 17 adds tokens equivalent to the token amount to the remaining token amount, and returns to the processing of step S21.

[0047] <Transport Transmitter Processing: First Example of Packet Transmission Processing> A first example of packet transmission processing by the transport transmitter 17 will be described with reference to FIG. As shown in FIG. 6, in step S30, the transport sending unit 17 determines whether the remaining token amount is equal to or greater than a preset threshold value (for example, 1). If the remaining token amount is equal to or greater than 1 (Yes in step S30), the transport transmission unit 17 proceeds to the process of step S32. If the remaining token amount is not equal to or greater than 1 (No in step S30), the transport transmission unit 17 proceeds to the process of step S31.

[0048] In step S31, the transport transmitter 17 waits for an interrupt in step S22 of Fig. 5. Then, when an interrupt occurs, the transport transmitter 17 proceeds to the process of step S32.

[0049] In step S32, the transport transmission unit 17 determines whether or not a transmission packet of priority data is queued in the priority data queue 15B. If the data is queued in the priority data queue 15B (Yes in step S32), the transport transmitting unit 17 proceeds to the process of step S35. If the data is not queued in the priority data queue 15B (No in step S32), the transport transmitting unit 17 proceeds to the process of step S33.

[0050] In step S33, the transport sending unit 17 determines whether or not a chunk transmission packet is queued in the chunk queue 15A. If the chunk is queued in the chunk queue 15A (Yes in step S33), the transport transmission unit 17 proceeds to the process of step S35. If the chunk is not queued in the chunk queue 15A (No in step S33), the transport transmitting unit 17 proceeds to the process of step S34.

[0051] In step S34, the transport transmission unit 17 determines whether or not a transmission packet of non-priority data is queued in the non-priority data queue 15C. If the data is queued in the non-priority data queue 15C (Yes in step S34), the transport transmission unit 17 proceeds to the process of step S35. If the data is not queued in the non-priority data queue 15C (No in step S34), the transport transmission unit 17 returns to the process of step S30.

[0052] In step S35, the transport transmitter 17 acquires a transmission packet from the queue in which the transmission packet is queued. At this time, the transport transmitter 17 may write or update timestamp information as necessary. For example, in MMT, the MMT protocol header section has a delivery timestamp (transmission timestamp) field that indicates the transmission time of the packet. Therefore, when MMT is used as the multiplexing method for transmission packets, the transport transmitter 17 updates this delivery timestamp so that the time of sending from the transport transmitter 17 to the transmission path 2 is written.

[0053] In step S36, the transport sending unit 17 sends the transmission packet acquired in step S35 to the receiving device 3. Then, the transport sending unit 17 subtracts the token amount of the transmission packet length / maximum packet length from the remaining token amount, and returns to the processing of step S30.

[0054] In this way, in the packet transmission process of the first example, the transport transmission unit 17 transmits transmission packets in the order of priority: the priority data queue 15B, the chunk queue 15A, and the non-priority data queue 15C.

[0055] <Transport Transmitter Processing: Second Example of Packet Transmission Processing> 7, a second example of packet transmission processing by the transport transmitter 17 will be described. The packet transmission processing of this second example differs from the first example in that a counter is used to reduce the frequency of referring to the priority data queue 15B.

[0056] 7, in step S40, the transport transmission unit 17 sets a counter to N-1, where N can be set in advance to any value. In step S41, the transport transmission unit 17 determines whether the remaining token amount is equal to or greater than a preset threshold value (for example, 1). If the remaining token amount is equal to or greater than 1 (Yes in step S41), the transport transmission unit 17 proceeds to the process of step S43. If the remaining token amount is not equal to or greater than 1 (No in step S41), the transport transmission unit 17 proceeds to the process of step S42.

[0057] In step S42, the transport transmitter 17 waits for an interrupt in step S22 of Fig. 5. Then, when an interrupt occurs, the transport transmitter 17 proceeds to the process of step S43.

[0058] In step S43, the transport transmission unit 17 determines whether the counter is 0 or not. If the counter is 0 (Yes in step S43), the transport transmission unit 17 proceeds to the process of step S44. If the counter is not 0 (No in step S43), the transport transmitter 17 proceeds to the process of step S46.

[0059] In step S44, the transport transmission unit 17 determines whether or not a transmission packet of priority data is queued in the priority data queue 15B. If the data is queued in the priority data queue 15B (Yes in step S44), the transport transmitting unit 17 proceeds to the process of step S45. If the data is not queued in the priority data queue 15B (No in step S44), the transport transmitting unit 17 proceeds to the process of step S46.

[0060] In step S45, the transport transmitter 17 sets the counter to N-1. In step S46, the transport sending unit 17 determines whether or not a chunk transmission packet is queued in the chunk queue 15A. If the chunk is queued in the chunk queue 15A (Yes in step S46), the transport transmitting unit 17 proceeds to the process of step S49. If the chunk is not queued in the chunk queue 15A (No in step S46), the transport transmitting unit 17 proceeds to the process of step S47.

[0061] In step S47, the transport transmitter 17 sets the counter to zero. In step S48, the transport transmission unit 17 determines whether or not a transmission packet of non-priority data is queued in the non-priority data queue 15C. If the data is queued in the non-priority data queue 15C (Yes in step S48), the transport transmitting unit 17 proceeds to the process of step S49. If the data is not queued in the non-priority data queue 15C (No in step S48), the transport transmission unit 17 returns to the process of step S41.

[0062] In step S49, the transport sending unit 17 acquires the transmission packet from the queue in which the transmission packet is queued. At this time, the transport sending unit 17 may write or update timestamp information as necessary.

[0063] In step S50, the transport sending unit 17 sends the transmission packet acquired in step S49 to the receiving device 3. Then, the transport sending unit 17 subtracts the token amount of the transmission packet length / maximum packet length from the remaining token amount. In step S51, the transport transmitter 17 subtracts 1 from the counter and returns to the processing of step S41.

[0064] In the packet transmission process of the first example, if the priority data queue 15B becomes clogged, the state in which chunks cannot be transmitted continues. In contrast, in the packet transmission process of the second example, the transport transmitter 17 refers to the priority data queue 15B only once for N packet transmissions. As a result, in the packet transmission process of the second example, even if the priority data queue 15B becomes clogged, it is possible to prevent the state in which chunks cannot be transmitted.

[0065] As shown in Figures 8(a) and 8(b), the transmitting device 1 can output coded data of a video signal without using a CPB and complete a chunk with metadata attached as soon as the coding is completed. This prevents CPB delays within the transmitting device 1 and allows chunk data to be transmitted using the entire bandwidth of the broadcast transmission path, prioritizing it over non-priority data. This allows chunk data to be sent in the shortest possible time. However, it is possible that the data volume of a chunk containing an I-picture is large, causing delays in the transmission of subsequent chunks. Even in this case, as shown in Figures 9(a) and 9(b), the transmitting device 1 sets the maximum CPB delay to the GOP length, preventing accumulated delays exceeding the GOP length and ensuring real-time performance. Note that Figures 8(a) and 9(a) show the amount of coded data output by the encoding processing unit 11A. Also, Figures 8(b) and 9(b) show the amount of coded data output by the transmitting device 1.

[0066] [Configuration of receiving device] The configuration of the receiving device 3 according to the embodiment will be described with reference to FIG. As shown in FIG. 10, the receiving device 3 receives the multiplexed signal from the transmitting device 1, and includes a relay unit 3A and a video decoding unit 3B.

[0067] The relay unit 3A relays the video signal to the video decoding unit 3B, which will be described later, and includes a demultiplexing unit 30, a manifest file generating unit 31, a storage unit 32, and a server unit (reading unit) 33.

[0068] The demultiplexing unit 30 separates chunks made up of predetermined control information and encoded video signals from the multiplexed signal received by the relay unit 3 A. The demultiplexing unit 30 then outputs the separated control information to the manifest file generating unit 31 and writes the chunks to the storage unit 32.

[0069] The manifest file generation unit 31 generates a manifest file in which chunk location information (e.g., URL) and pointer information indicating a playback start chunk are described, in accordance with the control information separated by the demultiplexing unit 30. The manifest file generation unit 31 then writes the generated manifest file to the storage unit 32. Details of the manifest file generation unit 31 will be described later.

[0070] This manifest file is associated with a broadcast channel and describes chunk location information. For example, the manifest file describes a list of chunk file URLs or a chunk file template URL as location information. Furthermore, in the case of a template URL, an initial sequence number is described as pointer information identifying the playback start chunk to indicate the playback start chunk. For example, a chunk file template URL is "chunk_$Number$.m4s". The sequence number is used to generate the chunk file URL and is substituted for "$Number$" in the template URL, for example. For example, the manifest file format can be m3u8 for the HLS format or MPD for the MPEG-DASH format. Note that the manifest file format is not limited to the above.

[0071] The storage unit 32 is a storage device such as a memory or HDD (Hard Disk Drive) that stores chunks and a manifest file. Here, every time the relay unit 3A receives a new chunk, the manifest file generation unit 31 updates the contents of the manifest file in the storage unit 32. Therefore, the storage unit 32 stores the latest manifest file.

[0072] The server unit 33 reads out chunks and manifest files from the storage unit 32. The server unit 33 then outputs the read out chunks and manifest files to the video decoding unit 3B. Details of the server unit 33 will be described later.

[0073] Video decoding unit 3B requests (demands) a manifest file from server unit 33, acquires the manifest file from server unit 33 in response to the request, and requests and acquires chunks described in the acquired manifest file from server unit 33, thereby decoding the video signal. For example, video decoding unit 3B decodes the video signal using the same video encoding method as encoding processing unit 11A in FIG. 2.

[0074] <Processing in the server unit and video decoding unit> The processing of the server unit 33 and the video decoding unit 3B will be described with reference to FIG. 11, in step S60, the video decoding unit 3B specifies the URL of the manifest file stored in the storage unit 32 and requests chunks from the server unit 33. For example, the chunk request is made using HTTP (Hypertext Transfer Protocol) / TCP (Transmission Control Protocol).

[0075] The method for specifying the URL of the manifest file is arbitrary. For example, the URL of the manifest file can be specified by various methods, such as manual input by the user of the receiving device 3, specification by a QR code (registered trademark), or acquisition from a communication network. Also, by making the URL of the manifest file for live viewing constant for each channel, it is possible to view the intended channel live by requesting the URL of the manifest associated with the channel.

[0076] In step S61, the server unit 33 reads out the manifest file from the storage unit 32 and returns the manifest file to the video decoding unit 3B as a response. The video decoding unit 3B acquires the manifest file from the server unit 33.

[0077] In step S62, the video decoding unit 3B requests A chunks (for example, A=3) for the initial buffer from the server unit 33 in accordance with the acquired manifest file. In step S63, the server unit 33 reads the chunk from the storage unit 32 and returns a response to the video decoding unit 3B. In step S64, the video decoding unit 3B buffers A chunks.

[0078] In step S65, once A chunks have been buffered, the video decoding unit 3B decodes the video signal and plays back the video. Thereafter, the video decoding unit 3B requests the subsequent chunks and continues playing back the video while buffering the chunks (steps S66 to S68).

[0079] Note that the video decoding unit 3B may periodically request the manifest file from the server unit 33 to reacquire the latest information. In particular, if the chunk location information described in the manifest file is a list of chunk URLs that are not templates, the manifest file can be reacquired to acquire the URL of the subsequent chunk file. On the other hand, if the chunk location information is a template URL, the URL of the subsequent chunk file can be generated by incrementing (adding) the pointer information, so playback can generally be continued without periodically reacquiring the manifest file.

[0080] It should be noted that an HTTP (Hypertext Transfer Protocol) server can be used as the server unit 33. A typical HTTP server responds with an error message (NotFound error) indicating that the specified file does not exist if the file at the URL requested by GET does not exist. In other words, if the requested chunk is incomplete, a NotFound error is returned to the video decoding unit 3B. Therefore, the video decoding unit 3B can simply repeat the request at the same URL at regular intervals until the chunk can be acquired (until the NotFound error is no longer returned).

[0081] Furthermore, the server unit 33 may withhold a response to a chunk acquisition request from the video decoding unit 3B until the chunk is completed, and may respond after the chunk is completed. Furthermore, the server unit 33 may respond sequentially from the portion of the chunk that has been received, even if the entire chunk is incomplete. For example, in HTTP, by using Chunked Transfer Encoding, it is possible to respond sequentially with data that is not entirely complete.

[0082] <Details of the manifest file generator> The following provides additional details about the manifest file generation unit 31. The manifest file generation unit 31 updates the manifest file each time a chunk is completed so that the chunk that the video decoding unit 3B should acquire first, including the chunk for the initial buffer, is indicated by the top of the list of chunk file URLs or pointer information. Here, the sequence number of the chunk currently being received is assumed to be B. In this case, the manifest file generation unit 31 writes the chunk file URL or template URL and the sequence number (BA), which is pointer information indicating the start of playback, in the manifest file so that the video decoding unit 3B can acquire the chunk (sequence number = BA) that is A chunks before, which is the initial buffer count.

[0083] This allows video decoding unit 3B to acquire A chunks, which is the initial buffer number, without waiting for the chunk to be received (sequence number = B), and then immediately start playing video from the chunk with sequence number = BA. Then, video decoding unit 3B starts playing video from the chunk with sequence number = BA, and when it has played one chunk of video, it requests the chunk with sequence number = B to buffer the chunk. At this time, there is a high probability that video decoding unit 3B has already received the chunk with sequence number = B, so errors in response to requests are reduced and chunks can be acquired smoothly without requiring a long response time.

[0084] (Variation) Although the embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design modifications and the like within the scope of the present invention. In the above-described embodiment, the video encoding unit of the transmitting device is described as receiving the maximum CPB delay and the CBR bit rate value as parameters, but this is not limited to this. For example, the video encoding unit may receive the CPB size as a parameter instead of the maximum CPB delay. Furthermore, in addition to the maximum CPB delay or the CPB size, an additional parameter such as an initial buffer capacity (the encoding process is started assuming that a portion of the CPB capacity is being used as the initialization state) may also be received. These various parameters are parameters for managing the CPB state in detail and performing advanced rate control, and do not significantly change the basic operation of the CPB and do not impair the feasibility of the present invention.

[0085] In the above-described embodiment, when MMT is used as the multiplexing method, the control information generation unit of the transmitting device generates and transmits an MPT as control information, but this is not limited to this. For example, even when MMT is used as the multiplexing method, an MPD, which is a manifest file defined in MPEG-DASH, can be multiplexed using a general-purpose data transmission method or the like and transmitted as control information. In this case, the manifest file generation unit of the receiving device may generate an MPD or a manifest file in another format based on the MPD demultiplexed from the MMT.

[0086] In the above embodiment, the relay unit and the video decoding unit are integrated in the receiving device, but the present invention is not limited to this. For example, in the receiving device, the relay unit may be a relay device such as a home gateway, and may be independent from the video decoding unit.

[0087] In the above-described embodiment, the transmitting device and the receiving device are described as independent pieces of hardware, but the present invention is not limited to this. For example, the present invention can also be realized by a program that causes hardware resources such as a CPU, memory, and hard disk of a computer to function as the transmitting device or receiving device. This program may be distributed via a communication line or written to a recording medium such as a CD-ROM or flash memory.

[0088] (Reference example) A receiving device 4 according to a reference example will be described with reference to FIG. As shown in FIG. 12, the receiving device 4 receives a multiplexed signal from the transmitting device 1 via the transmission path 2, and includes a relay unit 4A and a video decoding unit 4B.

[0089] The relay unit 4A relays the video signal to the video decoding unit 3B, which will be described later, and includes a demultiplexing unit 40, a control information analyzing unit 41, and a CBR compensation buffer .

[0090] The demultiplexing unit 40 separates chunks, each of which is made up of control information describing a CBR and an encoded video signal, from the multiplexed signal received by the relay unit 4A. The demultiplexing unit 40 then outputs the separated control information to a control information analyzer 41. Furthermore, the demultiplexing unit 40 reconstructs coded data of the video signal from the separated chunks as necessary and outputs the reconstructed coded data to a CBR compensation buffer 42. For example, if coded data transmitted as chunks in a CMAF format or the like does not fit into the input of the video decoding unit 4B, the demultiplexing unit 40 may reconstruct the coded data into non-chunked coded data that can be processed by the video decoding unit 4B. The demultiplexing unit 40 may also perform conversion processing from the MMT format to a multiplexing format such as MPEG-2 TS (Transport Stream).

[0091] The control information analyzer 41 analyzes the control information input from the demultiplexer 40 and reads out the CBR bit rate value. That is, the control information analyzer 41 obtains the CBR described in the control information and outputs the obtained CBR to the CBR compensation buffer 42.

[0092] The CBR compensation buffer 42 is a buffer that smooths the coded data of the video signal using CBR, with reference to the CBR bit rate value input from the control information analysis unit 41. Then, the CBR compensation buffer 42 outputs the coded data smoothed using the CBR bit rate value to the video decoding unit 4B.

[0093] The video decoding unit 4B decodes and reproduces the coded data of the video signal input from the CBR compensation buffer 42. For example, the video decoding unit 4B decodes the video signal using the same video coding method as the coding processing unit 11A in FIG.

[0094] In the receiving device 4 according to the reference example, the relay unit 4A and the video decoding unit 4B are described as being integrated together, but this is not limiting. For example, in the receiving device 4, the relay unit 4A may be a relay device such as a home gateway, and may be independent from the video decoding unit 4B. [Explanation of symbols]

[0095] 1. Transmitting device 2 Transmission Line 3,4 Receiving device 3A, 4A relay section 3B,4B video decoding section 10 Audio coding unit 11 Video Encoding Unit 11A Encoding processing unit 11B Virtual CPB 11C Encoding control unit 12 Chunk Configuration Section 13 Control information generation unit 14 Transport multiplexing unit (multiplexing unit) 15A Chunk Queue 15B Priority Data Queue 15C Non-priority Data Queue 16 Token output unit 17 Transport Transmitter (Transmitter) 30,40 Demultiplexer 31 Manifest file generator 32 Storage section 33 Server unit (read unit) 41 Control Information Analysis Unit 42 CBR compensation buffer

Claims

1. A transmitting device that receives a video signal, encodes the input video signal, and transmits the encoded video signal, a video encoding unit that performs bit rate control using a virtual CPB on the video signal and outputs the encoded video signal at a bit rate equal to or higher than a predetermined CBR without using a CPB; a chunk constructing unit that constructs chunks from the video signal output by the video encoding unit at a bit rate equal to or higher than the CBR and adds metadata to the constructed chunks as header information; a control information generating unit that generates predetermined control information; a multiplexing unit that multiplexes the chunks to which the metadata is added and the control information into packets; a transmitter that transmits, as a multiplexed signal, prioritized data including the control information having a higher priority than the chunks, the chunks, and non-priority data having a lower priority than the chunks, so that the transmission rate is equal to or less than a preset transmission rate; A transmitting device comprising:

2. a token output unit that outputs tokens corresponding to a preset token amount at predetermined intervals; The transmission unit Add the amount of tokens corresponding to the tokens to the remaining amount of tokens; If the remaining token amount is equal to or greater than a preset threshold, transmitting the prioritized data, the chunk, and the non-priority data in order of priority; 2. The transmitting device according to claim 1, wherein a token amount corresponding to a transmission data amount of the priority data, the chunk, or the non-priority data is subtracted from the remaining token amount.

3. a priority data queue for storing the priority data; a chunk queue for queuing the chunks; a non-priority data queue to which the non-priority data is input and which stores the input non-priority data, the multiplexing unit stores the chunks constructed by the chunk construction unit in the chunk queue, and stores the control information generated by the control information generation unit in the priority data queue as the priority data; The transmitting device according to claim 1 or claim 2, characterized in that the transmitting unit transmits priority data queued in the priority data queue, chunks queued in the chunk queue, and non-priority data queued in the non-priority data queue in order of priority.

4. a voice encoding unit that receives a voice signal and encodes the received voice signal, The transmitting device according to claim 1, characterized in that the chunk construction unit constructs the chunk from a video signal output by the video encoding unit at a bit rate equal to or higher than the CBR and an audio signal encoded by the audio encoding unit.

5. A receiving device for receiving a multiplexed signal from the transmitting device according to claim 1, a demultiplexing unit that separates chunks made up of predetermined control information and the encoded video signal from the multiplexed signal; a manifest file generation unit that generates a manifest file in which location information of the chunks is described in accordance with the control information separated by the demultiplexing unit; a storage unit that stores the chunks and the manifest file; a reading unit that reads the chunks and the manifest file from the storage unit; a video decoding unit that requests the reading unit for the manifest file, acquires the manifest file from the reading unit in response to the request, receives chunks indicated by location information of the acquired manifest file in chronological order, and decodes a video signal; A receiving device comprising:

6. A program for causing a computer to function as the transmitting device according to claim 1.

7. A program for causing a computer to function as the receiving device according to claim 5.

Images