Method and device for controlling transmission of video stream

Abstract

To provide a method and a video stream shaper for controlling transmission of a video stream to prevent transmission bitrate overload in a network.SOLUTION: A computer-implemented method (100) for controlling transmission of a video stream includes estimating (110) the number of bits for a group of pictures (GOP) of a video stream to be transmitted, setting (120) a latency requirement for transmission of the video stream, determining (130) an average minimum video bitrate over the GOP on the basis of the estimated number of bits and a time corresponding to the time period represented by the duration of the GOP, setting (140) an output bitrate for transmission of the video frames on the basis of the latency requirement and the average minimum video bitrate for the video frames in the GOP, and sending (150) the video frame using the output bitrate.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure pertains to methods and devices for controlling the transmission of video streams.

Background Art

[0002] In video coding, a group of pictures, or GOP structure, specifies the order in which intra-frames and inter-frames are arranged. A GOP is a set of consecutive pictures within an encoded video stream. Each encoded video stream consists of consecutive GOPs from which visible frames are generated. Encountering a new GOP in a compressed video stream means that the decoder does not need the previous frame to decode the next frame, enabling fast seeking across the video. A GOP starts with an intra-frame and ends with the last frame in the group before the next intra-frame in the encoded video stream. The size of an intra-frame is generally significantly larger than that of an inter-frame, typically 10 to 50 times larger. Thus, transmitting an encoded video stream over a network results in a relatively high temporary transmission bitrate while an intra-frame is being transmitted, but a relatively low transmission bitrate while subsequent inter-frames are being transmitted. Thereby, bitrate spikes typically occur while transmitting an intra-frame. Modern networks and Internet connections can generally handle such bitrate spikes without further compressing the video stream. However, when multiple video cameras are installed in a particular network, although the overall data transmission bitrate has sufficient bandwidth on average for multiple video cameras, if multiple bitrate spikes occur simultaneously, the maximum data transmission bitrate of the network can be instantaneously exceeded. This can result in data packet loss, reduced video quality, etc. Therefore, it is necessary to prevent such transmission bitrate overload in the network.

Summary of the Invention

[0003] Therefore, an object of the present invention is to provide a method for controlling the transmission of a video stream to prevent transmission bitrate overload in a network.

[0004] According to a first aspect, a computer implementation method for controlling the transmission of a video stream is provided. The method includes estimating the number of bits for a picture group (GOP) of a video stream to be transmitted. The method further includes setting latency requirements for the transmission of the video stream. The method further includes determining the average minimum video bitrate across the GOP based on the estimated number of bits and the time corresponding to a time period represented by the duration of the GOP. The method further includes setting an output bitrate for transmitting video frames in the GOP based on the latency requirements and the average minimum video bitrate, and transmitting the video frames using the output bitrate.

[0005] Traditionally, surveillance networks were closed systems with permanently installed cables and guaranteed transmission capacity, particularly due to predictable bitrate curves. This method can enable data from surveillance cameras to be transmitted over networks where other types of traffic may also be transmitted. This may be even more preferable when long-distance transmission of video data occurs over links where transmission capacity cannot be guaranteed.

[0006] Compared to typical data traffic shapers, where the maximum bitrate is set and fixed, this disclosure provides a dynamic data traffic smoother that provides a reduced load on the network through which data is transmitted. Furthermore, the rate of data transmission between other devices in the network may be facilitated by the aforementioned possibility of reducing the load on the network, as described above.

[0007] Throughout this text, bitrate refers to the number of bits transmitted per second over a digital network. The bitrate curve refers to the bitrate as a function of time. Bitrate spikes in the bitrate curve refer to relatively short time intervals in the bitrate curve where the bitrate is significantly higher than the average bitrate, for example, the time interval over which intraframes are transmitted. The network referred to, unless otherwise specified, is any type of data transmission network (and possibly wireless) over which the transmission of the video frames under consideration occurs. The network can be any type of network for communicating digital information, such as a local network or a public network. The expressions “transmitting” and “pushing” data over a network can be used interchangeably throughout this text.

[0008] The average minimum video bitrate across a GOP is defined as the amount of data required to be transmitted during each frame period so that all video data corresponding to the GOP is transmitted during the GOP period. The frame period can be approximately 30-40 ms, for example, assuming a GOP length of 30 frames captured at a frame rate of 30 frames per second.

[0009] This method can facilitate the smoothing of the GOP's bitrate curve. Bitrate curve smoothing as used herein refers to a substantially constant transmit bitrate, which is the average bitrate along the GOP's transmission. This can prevent the video stream from momentarily exceeding the maximum bitrate of the network through which it is transmitted. Therefore, for example, the risk of information loss during a live stream can be reduced.

[0010] Throughout this explanation, latency refers to a time interval, where the start of the time interval is the timestamp at which the video frame is captured or generated, and the end of the time interval is the timestamp at which the video frame is transmitted. Thus, latency requirements can refer to the largest acceptable time interval of such an interval. This method can facilitate short latency times without compromising video quality, which can be advantageous, for example, in live streams.

[0011] Bit depth estimation may involve statistically analyzing the bit depth of one or more preceding GOPs in the video stream. This can facilitate accurate estimation of the bit depth of the GOP to be transmitted. This can further enhance the dynamic nature of data traffic smoothing when the instantaneous minimum bitrate required to transmit the upcoming GOP is continuously monitored and updated.

[0012] Estimating the number of bits may further involve buffering the video data corresponding to the GOP to be transmitted and analyzing the buffered video data.

[0013] The estimation of the bit depth may further involve determining the bit depth of the already encoded sequence of the video stream corresponding to the GOP. Thus, the already encoded sequence of the video stream can be smoothed and transmitted. If low latency is not a major concern, this may be advantageous, as an essentially exact average bitrate can be calculated from the captured and stored GOP. Therefore, this may further provide an improved estimation of the bit depth of the GOP.

[0014] This method may further include updating the average minimum video bitrate over the remaining untransmitted frames of the GOP based on the actual number of bits in the frames already transmitted in the GOP while video frames are being transmitted in the GOP. This allows the average minimum video bitrate to be updated according to real-time conditions in the scene being captured by the video camera. Therefore, dynamically updating the average minimum bitrate can be done while video frames are being transmitted in the GOP, which can further help avoid network overload. This may be even more preferable when the GOP time length, i.e., the time interval between transmissions of GOPs, is relatively long. Alternatively or additionally, this may reduce the average minimum video bitrate from the current GOP's transmission bitrate if preceding GOPs are experiencing rapid changes in the captured scene, while the current GOP is experiencing a relatively static captured scene.

[0015] A second embodiment provides a computer implementation method for controlling the simultaneous transmission of multiple video streams, the method comprising individually performing the method according to the features of the first embodiment for each of the multiple video streams.

[0016] When intra-frames of multiple video streams are transmitted simultaneously, network overload can occur due to the addition of multiple resulting bitrate spikes. This method can facilitate the smoothing of the data traffic to be transmitted, thereby avoiding such network overload. Consequently, data packet loss or reduction in video resolution, etc., can be avoided.

[0017] A method according to a second embodiment may further include setting the latency requirements for each transmission of a plurality of video streams individually, such that the latency requirements differ for each of the plurality of video streams. This may be advantageous when different types of video cameras among a plurality of video cameras are used, and / or when different resolutions, GOP lengths, etc., are used in video cameras among a plurality of video cameras of the same type. This may facilitate an improvement in the dynamic nature of the method.

[0018] The method according to the second embodiment may further include individually setting latency requirements for the transmission of each of the multiple video streams based on the type of each video stream. The types of video streams may be live streams, non-live streams, etc. Different video resolutions may be applicable to different types of video streams. For example, non-live streams may have a higher video resolution than live streams. In addition to higher video resolution, the individual setting of latency requirements may allow the transmission bitrate of non-live streams to be set essentially constant, which can prevent data loss. Thus, latency may be significantly higher for non-live streams compared to live streams. Furthermore, network bandwidth may be utilized more efficiently, thereby allowing a larger number of such non-live streams to be stacked for simultaneous transmission over the network.

[0019] According to a third aspect, a non-temporary computer-readable storage medium is provided that stores instructions for implementing the method according to the first and / or second aspect when executed on a device having processing capabilities.

[0020] The features and advantages described above relating to the first and second embodiments also apply to this third embodiment, where applicable. Therefore, references to the above are made to avoid excessive repetition.

[0021] According to a fourth aspect, a video stream shaper for controlling the transmission of a video stream, wherein the video stream shaper is A data size estimation function configured to estimate the number of bits for picture groups (GOPs) of the video stream to be transmitted, A latency setting function configured to set latency requirements for transmitting video streams, A bitrate determination function configured to determine the average minimum video bitrate across a GOP based on the estimated number of bits and the time period corresponding to the duration of the GOP, An output bitrate setting function configured to set the output bitrate for transmitting video frames in a GOP based on latency requirements and average minimum bitrate, A transmit function configured to send video frames using the output bitrate and A video stream shaper is provided, comprising a circuit configured to perform the following:

[0022] The features and advantages of the first embodiment described above also apply to this second embodiment, where applicable. Therefore, references to the above are made to avoid excessive repetition.

[0023] The data size estimation function may be further configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

[0024] The data size estimation function can be further configured to buffer the video data corresponding to the GOP to be transmitted and to analyze the buffered video data.

[0025] The data size estimation function may be further configured to determine the number of bits for an already encoded sequence of video streams corresponding to the GOP.

[0026] The circuit may further comprise a bitrate update function configured to update the average minimum video bitrate across the remaining yet-to-be-transmitted frames of the GOP based on the actual number of bits of the already-transmitted frames of the GOP.

[0027] The circuit may further be configured to execute a main function configured to individually execute the above-described functions of the video stream shaper for each of a plurality of video streams.

[0028] The above-described features of the method are also applicable to this second aspect, when applicable. References to the above are made to avoid undue repetition.

[0029] Generally, all terms used in the claims should be construed according to their ordinary meaning in the art, unless explicitly defined otherwise herein. All references to "a / an / the [element, device, component, means, step, etc.]" should be construed openly as referring to at least one instance of the said element, device, component, means, step, etc., unless explicitly stated otherwise. None of the steps of any method disclosed herein need to be performed in the exact order disclosed, unless explicitly stated otherwise.

[0030] The above as well as additional objects, features, and advantages of the present invention will be better understood through the following illustrative and non-limiting detailed description of the preferred embodiments, with reference to the accompanying drawings. In the accompanying drawings, the same reference numbers will be used for like elements.

Brief Description of the Drawings

[0031] [Figure 1] FIG. schematically shows a flowchart of a method for controlling the transmission of a video stream. [Figure 2A] FIG. schematically shows the transmission of data traffic of a picture group (GOP) over time. [Figure 2B] This diagram schematically illustrates the transmission of data traffic within a Picture Group (GOP) over time. [Figure 2C] This diagram schematically illustrates the transmission of data traffic within a Picture Group (GOP) over time. [Figure 3] This diagram schematically illustrates a method for controlling the transmission of multiple video streams. [Figure 4] This is a very schematic diagram showing a video stream shaper for controlling the transmission of a video stream. [Modes for carrying out the invention]

[0032] Next, the present invention will be described more fully below with reference to the accompanying drawings illustrating currently preferred embodiments of the invention. However, the present invention can be embodied in many different forms and should not be construed as being limited to the embodiments described herein, rather these embodiments are provided for completeness and completeness and to fully convey the scope of the invention to those skilled in the art.

[0033] With respect to Figure 1, a flowchart of a computer implementation method 100 for controlling the transmission of a video stream is shown. Method 100 can be implemented on any suitable device. In a non-limiting example, method 100 can be implemented in a video camera, in particular a digital monitoring video camera. Other examples include a fixed or portable computer, a network gateway, etc. This method can be considered a so-called traffic shaper for data traffic to be transmitted over a network. Method 100 includes estimating the number of bits for a picture group (GOP) of the video stream to be transmitted. A GOP may have the following different picture types: - Intra-encoded picture, I-frame. A picture coded independently of all other pictures. Each GOP begins with an I-frame in the decoding order. I-frames are sometimes alternatively called keyframes. - Predictive coded picture, P-frame. A P-frame or I-frame is a picture that contains motion-compensated difference information for a previously coded picture. P-frames can be created by pixel-level analysis, block-level analysis, or vector analysis. - Dual predictive coded picture, B-frame. This is a picture that contains motion compensation difference information between the previously decoded picture (I-frame or P-frame) and the picture that follows the B-frame (I-frame of the P-frame).

[0034] P-frames and B-frames are often collectively referred to as interframes.

[0035] The number of bits in an intraframe is generally significantly greater than that of an interframe following an intraframe, since interframes generally only contain the difference to the intraframe. However, this may not be true when capturing scenes with a lot of movement. For example, in nighttime camera surveillance, where little or no change may occur in the scene being captured by the camera during a given time interval, the file size of interframes in the captured video stream sequence may be substantially negligible. Furthermore, in such situations, a relatively large GOP length and / or a low frame rate may be used, where the GOP length refers to the number of frames between two subsequent intraframes. However, to adequately capture rapid changes in the scene being captured by the camera in camera surveillance, a relatively short GOP length may be preferred, regardless of the current (and potentially low) activity in the scene being captured. Throughout this disclosure, the consideration of pictures in the GOP will be exclusively of I-frames and P-frames. Those skilled in the art will understand that other types of frames, namely B-frames, may be considered within the claims. Furthermore, while static GOP length is considered throughout this disclosure, those skilled in the art will understand that dynamic GOP length may be implemented to conserve bandwidth, facilitate data traffic across a network, and so on. When describing / considering the size of a video frame (I-frame or P-frame), it should be understood that references are made to the corresponding disk space required to store such video frames. Therefore, it should be understood that an I-frame being larger than a P-frame is equivalent to an I-frame requiring more disk space to store than a P-frame. The terms “size” and “bit size” may sometimes be used interchangeably.

[0036] Estimating the bit depth for a GOP in a video stream to be transmitted can be done by analyzing one or more already transmitted GOPs in the video stream. By analyzing multiple already transmitted GOPs, statistical trends based on the bit depth of each of the already transmitted GOPs can be used for a relatively accurate estimation of the bit depth for the GOP to be transmitted. Thus, the method may include statistically analyzing the bit depth for one or more preceding GOPs in the video stream. The statistical analysis of the bit depth for one or more preceding GOPs in the video stream can utilize probabilistic methods in that the bit rate can be selected such that the risk of latency is below a threshold probability. The threshold probability can preferably be relatively low. For example, if a camera captures a video sequence of a relatively static scene, the P-frames of a particular GOP in the video sequence are generally much smaller than the I-frames of that particular GOP. The statistical analysis can then estimate a suitable bit rate to be used in subsequent GOPs. The statistical analysis may further include determining trends in the scenes the camera captures. For example, this trend may take into account the increasing rate of change in motion within a scene, which can thereby result in larger P-frames. This, in turn, may optimize the method for smoothing the corresponding data traffic transmitted over the network.

[0037] Method 100 further includes setting latency requirements for the transmission of a video stream in 120. As described above, latency requirements are requirements relating to the time difference between the capture / generation of video frame data and the transmission of video frame data. Latency requirements may be a maximum latency that cannot be exceeded. Latency requirements may sometimes be referred to as latency. Thus, over time, increased and amplified errors and potentially induced unwanted latency caused by the estimated video bitrate across the GOP to be transmitted can be prevented. Latency requirements may be set by the user. Latency requirements may depend on the user experience, network capacity, the type of scene to be monitored, etc. For example, a camera that monitors a gate and is used for real-time human communication may require near real-time transmission of data, e.g., 30 ms latency. Conversely, for a camera that monitors a parking lot, the latency may be significantly higher, e.g., 0.5 s. The target time in which the traffic shaper should have transmitted all data for one or more GOPs may be calculated based on the frame rate and GOP length. Therefore, the target time can be considered a timestamp at which the subsequent I-frame is transmitted in real time.

[0038] Method 100 further includes determining the average minimum bitrate across the GOP in 130 based on the estimated number of bits and the time corresponding to the time period represented by the duration of the GOP. The average minimum bitrate across the GOP may depend on the video resolution. That is, for example, if low latency is desired in a live stream, the average minimum bitrate across the GOP in the live stream may be relatively low, especially for scenes that are relatively visually static and captured by a live stream camera. The average minimum video bitrate across the GOP may be defined based on how much data needs to be transmitted during each frame period so that all video data corresponding to the GOP is transmitted during the GOP period.

[0039] Method 100 further includes, for video frames in the GOP, setting an output bitrate for transmitting the video frame in 140 based on latency requirements and the average minimum video bitrate, and transmitting the video frame in 150 using the output bitrate. Here, the transmission of each video frame in the GOP is thus considered individually. The output bitrate may be updated based on the estimated number of bits and the target time in which pushing the frame to be transmitted should have been completed.

[0040] Figures 2A–2C schematically summarize Method 100. Here, a GOP 200 is illustrated, comprising one I-frame I1 and eight P-frames P1–P8, which should not be interpreted as limiting or confusing to those skilled in the art. That is, any suitable number of P-frames may be possible within the claims. In the absence of latency, the transmission of video frames may generally occur within any suitable time interval. Such time intervals may preferably be shorter than or equal to the frame rate-dependent time interval corresponding to the elapsed time between two subsequent captured frames, as qualitatively shown in Figures 2A–2C. If the time interval is significantly shorter than the frame rate-dependent time interval, the corresponding bitrate curve will have a narrow bitrate spike for each transmitted P-frame, despite the potentially insignificant size of each P-frame. Again, the I-frame is found to be significantly larger than each of the subsequent P-frames. Thus, this example suggests a relatively moderate change in the captured scene. A typical frame rate may be 30 frames per second (fps). The GOP length may be, for example, 30 frames or 60 frames. Therefore, in the case of a frame rate of 30 fps and a GOP length of 60 frames, the duration of the GOP is 2 seconds. As will be readily understood by those skilled in the art, other suitable frame rates and GOP lengths are possible within the claims.

[0041] Figure 2A schematically illustrates the normal transmission of data traffic on GOP200 over time, where the transmission bitrate exhibits relatively narrow spikes during the transmission of I-frames. In this toy example, it is assumed that GOP200 has one I-frame I1 and a group of P-frames 210, each consisting of eight P-frames P1-P8. The apparent "area" of each frame in GOP200 qualitatively indicates the relative bit size of the frame within the GOP. The frames shown in Figure 2A(i) can be considered data packets. Thus, each frame I1, P1-P8 can be considered a data packet to be pushed across the network. Pushing data packets in this manner is equivalent to having minimal latency, as each individual data packet is transmitted as soon as the corresponding video frame is captured. Generally, and as seen in this example, I-frame I1 has a significantly larger bit size compared to the subsequent P-frames P1-P8, where it is assumed that all P-frames P1-P8 have substantially similar bit sizes, although the bit sizes within a true group of P-frames can generally differ.

[0042] Figure 2B schematically illustrates the smoothing of GOP200 over time. GOP200 is smoothed according to method 100 when it is transmitted. Here, a first latency requirement LR1 is set and defines the time interval during which bits of the I-frame should be pushed over the network. The I-frame I1 shown in Figure 2A is here split into two data packets and pushed over the network during a time interval corresponding to pushing two frames, for example, I-frame I1 and the first P-frame P1 in Figure 2A. Thus, I-frame I1 is not interpreted as being split into two different frames, but rather as being split into two different data packets. When GOP200 is transmitted over the network and played back on the receiver side, the corresponding video sequence is played back at normal speed and delayed by a time similar to the latency requirement. Smoothing requires that the remaining P-frames P1-P8 of GOP200 be combined into a larger packet so that GOP200 can be transmitted before subsequent GOPs begin. In this example, the last data packet may be a data packet containing the seventh P7 and eighth P8 P-frames, so that a GOP comprising data packets I1a, I1b, P1-P6 and a larger P-frame packet PL1 comprising the seventh P7 and eighth P8 P-frames is pushed within the target time TG1. Any pair of P-frames, e.g., the first P-frame P1 and the second P-frame P2, etc., can be used equally well to constitute a similar data packet. This may result in a corresponding bitrate curve with a first bitrate peak 252 corresponding to the bitrate when pushing I-frame packets I1a and I1b over the network. Similarly, when pushing the larger P-frame packet PL1 over the network, a second bitrate peak 254 may occur in the bitrate curve 250. Generally, data packets can be about 1.5kB in size. A relatively large I-frame can be split into thousands of data packets.A frame can be transmitted by a data packet in at least three ways: (i) one frame in one data packet, (ii) multiple frames in one data packet, or (iii) one frame in multiple data packets. In principle, it may be even possible to split one number of frames into another number of data packets. For example, two data packets may contain a total of three frames, where each data packet contains data corresponding to 1.5 frames.

[0043] Figure 2C schematically shows another example of smoothed data corresponding to GOP200 over time, of data traffic that is smoothed according to method 100 when GOP200 is transmitted. Here, a second latency requirement LR2 is set, defining the time interval at which I-frames should be pushed over the network. The I-frame I1 shown in Figure 2A is here split into five data packets I1i-I1v of substantially equal bit sizes. P-frames P1-P8 are here combined into data packets, each containing two P-frames, i.e., a first P-frame packet containing the first P1 and second P2 P-frames, a second P-frame packet containing the third P3 and fourth P4 P-frames, and so on. This example may produce a corresponding bitrate curve 260 with substantially constant bitrate when pushing the I-frame packets I1i-I1v and subsequent P-frame packets. It should be understood that a larger time interval value for latency requirements may result in a more constant bitrate curve when pushing data packets corresponding to the bits of GOP200, and vice versa. The bitrate curve may thus deviate little to negligible from the average minimum bitrate of the GOP. Therefore, latency requirements can be considered a trade-off between the length of the time interval of the latency requirement and the smoothness of the corresponding bitrate curve. It should be understood that the example shown in Figure 2C may be relatively ideal in terms of network load. Since the estimation of the number of bits for the GOP of the video stream to be transmitted may depend on GOPs that have already been transmitted, fluctuations may occur that cause deviations in the average bitrate between two GOPs.

[0044] In Figures 2A to 2C, the frame rate may appear to be similar to the rate at which data packets corresponding to frames in GOP200 are transmitted. However, data packets can be transmitted equally well at lower rates, such that each data packet consists of multiple frames, and each data packet is sent over a time period corresponding to the capture of the frames contained within the data packet.

[0045] The estimated bit count of 110 may include buffering the video data corresponding to the GOP to be transmitted and analyzing the buffered video data. Thus, all frames in the GOP may be buffered, and then the total bit size of all frames, as well as the average bit rate required for the GOP to be transmitted within its target time, can be calculated. Following this, data packets may be pushed at a bit rate that matches the average bit rate of the GOP. In this scenario, relatively high latency may exist, but the bit rate may be further smoothed or even substantially constant while pushing data packets corresponding to the GOP across the network.

[0046] Alternatively, the estimation of the number of bits (110) may involve determining the number of bits for an already encoded sequence of video streams corresponding to the GOP. Therefore, smoothing of the already encoded video data may be possible. This may be preferable in the case of pre-buffering when triggered by an event, e.g., a change in the scene captured by a video camera, and therefore it is possible to push data packets corresponding to a certain past time interval. Such a past time interval could be, for example, 15 seconds ahead, a minute ahead, etc.

[0047] Method 100 may further include updating the average minimum video bitrate over the remaining untransmitted frames of the GOP based on the actual number of bits of the frames already transmitted in the GOP while transmitting video frames in the GOP. Thus, the average minimum video bitrate can be adjusted according to the real-time situation in the scene being captured by the video camera. This embodiment may be preferred when a preceding GOP is experiencing rapid changes in the captured scene, and the current GOP is experiencing a relatively static captured scene.

[0048] With respect to Figure 3, a computer implementation method 300 for controlling the simultaneous transmission of multiple video streams is shown. Method 300 includes individually performing Method 100 described above for each of the multiple video streams. Method 300 may include smoothing the video streams from multiple cameras so that the sum bitrate of the video streams is below the maximum allowable bitrate for data transmission over the network. If the corresponding latency requirements for each video stream are relatively short so that bitrate spikes occur for the multiple video streams, the second method may delay one or more video streams, if necessary, so that the sum bitrate of the multiple video streams is below the maximum allowable bitrate of the network.

[0049] Method 300 may further include setting the latency requirements for the transmission of each of the multiple video streams individually, such that the latency requirements are different for each of the multiple video streams.

[0050] Method 300 may further include setting individual latency requirements for the transmission of each of the multiple video streams based on the type of each video stream. The types of video streams may include live streams to be stored for later review. Playback of a live stream may be subject to a delay between capture and playback of several seconds or less. If the type of video stream is a live stream, the latency requirement may be a relatively short time period. This may then set constraints on the number of live streams being transmitted over the network. Method 300 may evaluate the activity in each scene associated with each live stream, thereby assigning less active scenes, in some cases, longer transient latency requirements. Alternatively, if the I-frames of multiple live streams are transmitted substantially simultaneously, one or more live streams may be delayed to some extent relative to each other.

[0051] Instructions for implementing the method described above are stored in a non-temporary computer-readable storage medium and executed on a device with processing capabilities.

[0052] With respect to Figure 4, a video stream shaper 400 for controlling the transmission of a video stream is shown in a very schematic manner. The video stream shaper 400 can be implemented on any suitable device. In a non-limiting example, the video stream shaper 400 can be implemented in a video camera, in particular a digital monitoring video camera. Other examples include a fixed or portable computer, a network gateway, etc. The above-described features relating to (one or more) methods 100, 300 also apply to the video stream shaper 400 when applicable. Therefore, references to the above are made to avoid excessive repetition.

[0053] The video stream shaper 400 includes a circuit 420.

[0054] Circuit 420 is configured to provide overall control over the functions and operation of the video stream shaper 400. Circuit 420 may include a processor, such as a central processing unit (CPU), microcontroller, or microprocessor. The processor is configured to execute program code stored in circuit 420 to perform the functions and operation of the video stream shaper 400.

[0055] The executable functions described below may be stored in memory. Memory may be one or more of the following: buffers, flash memory, hard drives, removable media, volatile memory, non-volatile memory, random-access memory (RAM), or other suitable devices. In a typical configuration, memory may include non-volatile memory for long-term data storage and volatile memory serving as system memory for circuit 420. Memory may exchange data with circuit 420 via a data bus. Accompanying control lines and an address bus between memory and circuit 420 may be present.

[0056] The functions and operations of circuit 420 may be embodied in the form of executable logic routines, such as computer code portions, software programs, etc., which are stored in a non-temporary computer-readable medium of the video stream shaper 400, for example, memory, and executed by circuit 420, for example, by using a processor. The functions and operations of the video stream shaper 400 may be a standalone software application or form part of a software application that performs additional tasks related to the video stream shaper 400. The functions and operations described may take into account how the corresponding device is configured to perform them. Also, while the functions and operations described may be implemented in software, such functions may also be performed through dedicated hardware or firmware, or some combination of hardware, firmware and / or software.

[0057] Circuit 420 is configured to perform a data size estimation function 421 configured to estimate the number of bits for a picture group (GOP) of a video stream to be transmitted. Circuit 420 further includes a latency setting function 422 configured to set latency requirements for transmitting the video stream. Circuit 420 further includes a bitrate determination function 423 configured to determine the average minimum video bitrate over the GOP based on the estimated number of bits and the time corresponding to a time period represented by the duration of the GOP. Circuit 420 further includes an output bitrate setting function configured to set the output bitrate for transmitting video frames in the GOP based on the latency requirements and the average minimum bitrate. Circuit 420 further includes a transmit function 425 configured to transmit video frames using the output bitrate.

[0058] The data size estimation function 421 may be further configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

[0059] The data size estimation function 421 may be further configured to buffer video data corresponding to the GOP to be transmitted and to analyze the buffered video data.

[0060] The data size estimation function 421 may be further configured to determine the number of bits for an already encoded sequence of video streams corresponding to the GOP.

[0061] The circuit 420 may further include a bitrate update function configured to update the average minimum video bitrate across the remaining untransmitted frames of the GOP based on the actual number of bits in the frames already transmitted by the GOP.

[0062] Circuit 420 may be further configured to perform a main function 426, which is configured to perform functions 421 to 425 individually for each of the multiple video streams, as described above.

[0063] Those skilled in the art will understand that the present invention is by no means limited to the preferred embodiments described above. In other words, many modifications and variations are possible within the scope of the appended claims.

[0064] For example, a particular video stream shaper 400 may control the transmission of two independent video streams. In such a situation, the latency requirements of one of the two video streams may take into account the properties of the other video stream to optimize the smoothing of the transmitted data. Those skilled in the art will understand that this may also apply to three or more independent video streams.

[0065] In addition, variations of the disclosed embodiments may be understood and realized by those skilled in the art when performing the claimed invention, based on the drawings, this disclosure, and the accompanying study of the claims.

Claims

1. A computer-implemented method (100) for controlling the transmission of a video stream, the method (100) comprising: estimating (110) the number of bits for a group of pictures (GOP) (200) of the video stream to be transmitted, the GOP comprising an intra-frame and one or more inter-frames, estimating the number of bits (110); setting (120) a latency requirement for the transmission of the video stream, the latency requirement being a requirement regarding the time difference between the capture / generation of video frame data and the transmission of the video frame data, setting the latency requirement (120); determining (130) an average minimum video bitrate over the GOP (200) based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP (200); for each video frame in the GOP (200), determining the number of data packets into which the video frame is to be spread or determining that the video frame is to be transmitted in the same data packet as another video frame of the GOP, and setting an output bitrate for the transmission of the video frame based on the latency requirement and the average minimum video bitrate (140); transmitting (150) the video frame using the output bitrate The method (100) comprising.

2. The method (100) according to claim 1, wherein estimating (110) the number of bits comprises statistically analyzing the number of bits for one or more preceding GOPs in the video stream.

3. The method (100) according to claim 1, wherein estimating (110) the number of bits comprises buffering video data corresponding to the GOP (200) to be transmitted and analyzing the buffered video data.

4. The method (100) according to claim 1, wherein estimating (110) the number of bits comprises determining the number of bits for an already encoded sequence of the video stream corresponding to the GOP (200).

5. During the transmission of the video frame in the GOP (200), further comprising updating the average minimum video bit rate across the remaining yet-to-be-transmitted frames of the GOP (200) based on the actual number of bits of the already-transmitted frames of the GOP (200), the method (100) according to claim 1.

6. A computer-implemented method (300) for controlling the simultaneous transmission of a plurality of video streams, the method (300) comprising individually executing the method according to claim 1 for each of the plurality of video streams, the computer-implemented method (300).

7. The method (300) according to claim 6, further comprising individually setting the latency requirements for the transmission of each of the plurality of video streams such that the latency requirements are different for each of the plurality of video streams.

8. The method (300) according to claim 6, further comprising individually setting the latency requirements for the transmission of each of the plurality of video streams based on the type of each of the plurality of video streams.

9. A non-transitory computer-readable storage medium storing instructions for implementing the method according to any one of claims 1 to 8 when executed on a device having processing capabilities.

10. A video stream shaper (400) for controlling the transmission of a video stream, the video stream shaper (400) comprising a data size estimation function (421) configured to estimate the number of bits for a picture group (GOP) (200) of the video stream to be transmitted, the GOP comprising an intra-frame and one or more inter-frames, the data size estimation function (421); a latency setting function (422) configured to set the latency requirements for the transmission of the video stream, the latency requirements being requirements related to the time difference between the capture / generation of video frame data and the transmission of the video frame data, the latency setting function (422); A bitrate determination function (423) configured to determine an average minimum video bitrate over the GOP (200) based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP (200); For each video frame in the GOP (200), an output bitrate setting function (424) configured to set an output bitrate for transmission of the video frame based on the latency requirement and the average minimum bitrate by determining the number of data packets to which the video frame is to be spread or determining that the video frame is to be transmitted in the same data packet as another video frame of the GOP; A transmission function (425) configured to transmit the video frame using the output bitrate; A video stream shaper (400) comprising a circuit (420) configured to execute the above.

11. The video stream shaper (400) according to claim 10, wherein the data size estimation function (421) is further configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

12. The video stream shaper (400) according to claim 10, wherein the data size estimation function (421) is further configured to buffer video data corresponding to the GOP (200) to be transmitted and analyze the buffered video data.

13. The video stream shaper (400) according to claim 10, wherein the data size estimation function (421) is further configured to determine the number of bits for an already encoded sequence of the video stream corresponding to the GOP (200).

14. The video stream shaper (400) according to any one of claims 10 to 13, wherein the circuit (420) further comprises a bitrate update function configured to update the average minimum video bitrate over the remaining yet untransmitted frames of the GOP (200) based on the actual number of bits of the already transmitted frames of the GOP (200).

Images