Method and device for controlling the transmission of video streams - Patents.com

Abstract

To provide a method and a video stream shaper for controlling transmission of a video stream to prevent transmission bitrate overload in a network.SOLUTION: A computer-implemented method (100) for controlling transmission of a video stream includes estimating (110) the number of bits for a group of pictures (GOP) of a video stream to be transmitted, setting (120) a latency requirement for transmission of the video stream, determining (130) an average minimum video bitrate over the GOP on the basis of the estimated number of bits and a time corresponding to the time period represented by the duration of the GOP, setting (140) an output bitrate for transmission of the video frames on the basis of the latency requirement and the average minimum video bitrate for the video frames in the GOP, and sending (150) the video frame using the output bitrate.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure pertains to methods and devices for controlling the transmission of video streams. [Background technology]

[0002] In video coding, a group of pictures, or GOP structure, specifies the order in which intra and inter frames are arranged. A GOP is a collection of consecutive pictures in a coded video stream. Each coded video stream consists of consecutive GOPs from which visible frames are generated. Encountering a new GOP in a compressed video stream means that the decoder does not need the previous frame to decode the next frame, allowing for fast seeking across the video. A GOP begins with an intra frame and ends with the last frame in the group before the next intra frame in the coded video stream. The size of an intra frame is generally significantly larger than the size of an inter frame, typically 10 to 50 times larger. Therefore, transmitting a coded video stream over a network results in a relatively large temporary transmission bit rate while an intra frame is transmitted, but a relatively low transmission bit rate while a subsequent inter frame is transmitted. As a result, bit rate spikes typically occur during the transmission of intra frames. Modern networks and Internet connections can generally handle such bitrate spikes without further compressing the video stream. However, when multiple video cameras are installed in a particular network, the overall data transmission bitrate may, on average, have sufficient bandwidth for the multiple video cameras, but may momentarily exceed the maximum data transmission bitrate of the network if multiple bitrate spikes occur simultaneously. This may result in lost data packets, reduced video quality, etc. Therefore, it is necessary to prevent such transmission bitrate overloads in the network. Summary of the Invention

[0003] It is therefore an object of the present invention to provide a technique for controlling the transmission of video streams in order to prevent transmission bitrate overload in a network.

[0004] According to a first aspect, a computer-implemented method for controlling transmission of a video stream is provided. The method includes estimating a number of bits for a group of pictures (GOP) of the video stream to be transmitted. The method further includes setting a latency requirement for the transmission of the video stream. The method further includes determining an average minimum video bitrate over the GOP based on the estimated number of bits and a time corresponding to a period of time represented by a duration of the GOP. The method further includes, for video frames in the GOP, setting an output bitrate for transmission of the video frames based on the latency requirement and the average minimum video bitrate, and transmitting the video frames using the output bitrate.

[0005] Traditionally, surveillance networks have been closed systems with permanently installed cables and guaranteed transmission capacity resulting from, among other things, predictable bit rate curves. The present method may allow data from surveillance cameras to be transmitted over networks where other traffic types may be transmitted. This may be even more preferable when long-distance transmission of video data occurs over links where transmission capacity may not be guaranteed.

[0006] Compared to typical data traffic shapers, where the maximum bit rate is set and fixed, the present disclosure provides a dynamic data traffic smoother that provides a reduced load on the network over which the data is transmitted. Furthermore, the transmission speed of data between other devices in the network can be accelerated due to the above possibilities of reducing the load on the network, as described above.

[0007] Throughout this text, bit rate refers to the number of bits per second being transmitted over a digital network. A bit rate curve refers to the bit rate as a function of time. A bit rate spike in a bit rate curve refers to a relatively short time interval on the bit rate curve where the bit rate is significantly greater than the average bit rate, e.g., the time interval during which an intra-frame is transmitted. The network referred to, unless otherwise specified, is any type of data transmission network (possibly wireless) over which the transmission of the considered video frames occurs. The network may be any type of network for communicating digital information, e.g., a local network, a public network, etc. The expressions "sending" and "pushing" data over a network may be used interchangeably throughout.

[0008] The average minimum video bitrate across a GOP is defined as the amount of data that needs to be transmitted during each frame period such that all video data corresponding to the GOP is transmitted during the GOP period. A frame period may be approximately 30-40 ms, assuming a GOP length of 30 frames captured at a frame rate of 30 frames per second, for example.

[0009] This method can facilitate smoothing of the bit rate curve of a GOP. Smoothing of the bit rate curve in this specification refers to a substantially constant transmission bit rate, which is the average bit rate along the transmission of a GOP. This can prevent the video stream from momentarily exceeding the maximum bit rate of the network through which it is transmitted. Therefore, for example, the risk of losing information in a live stream can be reduced.

[0010] Latency refers to a time interval throughout this description, where the start of the time interval is the timestamp at which a video frame is captured or generated, and the end of the time interval is the timestamp at which the video frame is transmitted. A latency requirement may thereby refer to a maximum allowable such time interval. This method may facilitate short latency times without compromising video quality, which may be advantageous, for example, in live streams.

[0011] The bit count estimation may include statistically analyzing the bit counts for one or more preceding GOPs in the video stream, which may facilitate proper estimation of the number of bits for the GOP to be transmitted, which may further facilitate the dynamic nature of data traffic smoothing as the instantaneous minimum bit rate required to transmit the upcoming GOP is continuously monitored and updated.

[0012] Estimating the number of bits may further include buffering video data corresponding to the GOP to be transmitted and analyzing the buffered video data.

[0013] The bit count estimation may further include determining the number of bits for an already-encoded sequence of the video stream corresponding to the GOP. Thus, the already-encoded sequence of the video stream can be smoothed and transmitted. If short latency is not very important, this may be advantageous because an essentially accurate average bit rate can be calculated from the captured and stored GOP. Thus, this may further provide an improved estimation of the number of bits for the GOP.

[0014] The method may further include updating the average minimum video bitrate over the remaining untransmitted frames of the GOP based on the actual number of bits of the already transmitted frames of the GOP during transmission of the video frames in the GOP. The average minimum video bitrate can thereby be updated according to real-time conditions in the scene captured by the video camera. Therefore, dynamically updating the average minimum bitrate can be performed during transmission of the video frames in the GOP, which may further facilitate avoiding network overload. This may be more preferable when the GOP duration, i.e., the time interval between GOP transmissions, is relatively long. Alternatively or additionally, this may reduce the transmission bitrate of the current GOP when the preceding GOP experiences rapid changes in the captured scene and the current GOP experiences a relatively static captured scene.

[0015] According to a second aspect, there is provided a computer-implemented method for controlling simultaneous transmission of multiple video streams, the method comprising individually performing, for each of the multiple video streams, a method according to the above-mentioned features of the first aspect.

[0016] When intraframes of multiple video streams are transmitted simultaneously, network overload may occur due to the addition of multiple bitrate spikes. This method may facilitate smoothing of data traffic to be transmitted, thereby avoiding such network overload. Therefore, data packet loss or reduction of video resolution may be avoided.

[0017] The method according to the second aspect may further include individually setting latency requirements for transmission of each of the multiple video streams, such that the latency requirements are different for each of the multiple video streams. This may be advantageous when different types of video cameras of the multiple video cameras are used and / or different resolutions, GOP lengths, etc. of multiple video cameras of the same type are used. This may help improve the dynamic nature of the method.

[0018] The method according to the second aspect may further include individually setting latency requirements for transmission of each of the multiple video streams based on the type of each of the multiple video streams. The type of video stream may be a live stream, a non-live stream, etc. Different video resolutions may be applicable for different types of video streams. For example, a non-live stream may have a larger video resolution than a live stream. In addition to the higher video resolution, the individual setting of latency requirements may allow the transmission bit rate of the non-live stream to be set essentially constant, which may prevent data loss. Thus, latency may be significantly greater for a non-live stream compared to a live stream. Furthermore, network bandwidth may be more efficiently utilized, thereby allowing a larger number of such non-live streams to be stacked for simultaneous transmission over the network.

[0019] According to a third aspect, there is provided a non-transitory computer-readable storage medium storing instructions for, when executed on a device having processing capability, implementing a method according to the first and / or second aspect.

[0020] The features and advantages described above in relation to the first and second aspects also apply to this third aspect, where applicable, and reference is therefore made thereto to avoid undue repetition.

[0021] According to a fourth aspect, there is provided a video stream formatter for controlling transmission of a video stream, the video stream formatter comprising: a data size estimation function configured to estimate a number of bits for a group of pictures (GOP) of the video stream to be transmitted; a latency setting function configured to set a latency requirement for transmission of the video stream; a bitrate determination function configured to determine an average minimum video bitrate over the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP; an output bitrate setting function configured to set an output bitrate for transmission of the video frames based on a latency requirement and an average minimum bitrate for the video frames in the GOP; a transmit function configured to transmit video frames using an output bit rate; A video stream formatter is provided, comprising circuitry configured to perform:

[0022] The features and advantages discussed above in relation to the first aspect also apply to this second aspect, where applicable, and reference is therefore made thereto to avoid undue repetition.

[0023] The data size estimation function may further be configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

[0024] The data size estimation function may be further configured to buffer video data corresponding to a GOP to be transmitted and analyze the buffered video data.

[0025] The data size estimation function may be further configured to determine a number of bits for an already encoded sequence of the video stream corresponding to the GOP.

[0026] The circuitry may further comprise a bitrate update function configured to update the average minimum video bitrate over the remaining yet to be transmitted frames of the GOP based on the actual number of bits of the already transmitted frames of the GOP.

[0027] The circuitry may be further configured to perform a main function configured to individually perform the above-described functions of the video stream formatter for each of the plurality of video streams.

[0028] The above-mentioned features of the method, when applicable, also apply to this second aspect, and in order to avoid undue repetition, reference is made to the above.

[0029] In general, all terms used in the claims should be interpreted according to their ordinary meaning in the art unless expressly defined otherwise herein. All references to "a / an / the [element, device, component, means, step, etc.]" should be openly interpreted as referring to at least one instance of said element, device, component, means, step, etc., unless expressly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless expressly stated.

[0030] The above, as well as additional objects, features and advantages of the present invention will be better understood through the following illustrative, non-limiting detailed description of preferred embodiments, with reference to the accompanying drawings, in which like reference numerals will be used for similar elements, in which: [Brief explanation of the drawings]

[0031] [Figure 1] 1 shows a schematic flow chart of a method for controlling the transmission of a video stream; [Figure 2A] 1 shows a schematic diagram of the transmission of data traffic of a group of pictures (GOP) over time; [Figure 2B] 1 shows a schematic diagram of the transmission of data traffic of a group of pictures (GOP) over time; [Figure 2C] 1 shows a schematic diagram of the transmission of data traffic of a group of pictures (GOP) over time; [Figure 3] FIG. 1 illustrates schematically a method for controlling the transmission of multiple video streams. [Figure 4] FIG. 2 shows very schematically a video stream formatter for controlling the transmission of a video stream. DETAILED DESCRIPTION OF THE INVENTION

[0032] The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which presently preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and to fully convey the scope of the invention to those skilled in the art.

[0033] With reference to FIG. 1 , a flowchart of a computer-implemented method 100 for controlling the transmission of a video stream is shown. Method 100 may be implemented on any suitable device. By way of non-limiting example, method 100 may be implemented in a video camera, in particular a digital monitoring video camera. Other examples are fixed or portable computers, network gateways, etc. The method may be considered as a so-called traffic shaper for data traffic to be transmitted over a network. Method 100 includes, at 110, estimating the number of bits for a group of pictures (GOP) of the video stream to be transmitted. A GOP may comprise different picture types: - Intra-coded pictures, I-frames. These are pictures that are coded independently of all other pictures. Each GOP begins, in decoding order, with an I-frame. I-frames are sometimes alternatively called key frames. - Predictively coded picture, P-frame. A picture that contains motion compensation difference information relative to a previously decoded picture, which can be a P-frame or an I-frame. P-frames can be created by pixel-level analysis, block-level analysis, or vector analysis. - Bi-predictive coded pictures, B-frames, which are pictures that contain motion compensation difference information relative to the previously decoded picture (I-frame or P-frame) and the picture that follows the B-frame (I-frame of a P-frame).

[0034] P-frames and B-frames are often collectively referred to as inter-frames.

[0035] The number of bits in an intraframe is generally significantly larger than the number of bits in an interframe following an intraframe, because the interframe generally contains only the differences relative to the intraframe. However, this may not be the case when capturing a scene with a lot of movement. Thus, for example, in nighttime camera surveillance, where there may be little or no change in the scene being captured by the camera during a certain time interval, the file size of interframes in the captured video stream sequence may be substantially negligible. Furthermore, in such situations, a relatively large GOP length and / or a low frame rate may be used, where GOP length refers to the number of frames between two subsequent intraframes. However, to adequately capture rapid changes in the scene being captured by the camera in camera surveillance, a relatively short GOP length may be preferred regardless of the current (possibly low) activity in the scene being captured. Throughout this disclosure, discussion of pictures in a GOP refers exclusively to I-frames and P-frames. Those skilled in the art will understand that other types of frames, i.e., B-frames, may be considered within the scope of the claims. Additionally, while static GOP lengths are considered throughout this disclosure, those skilled in the art will appreciate that dynamic GOP lengths may be implemented to conserve bandwidth, expedite data traffic over a network, etc. It should be appreciated that when describing / discussing the size of a video frame (I-frame or P-frame), reference is made to the corresponding disk space required to store such a video frame. Thus, it should be understood that an I-frame being larger than a P-frame is equivalent to an I-frame requiring more disk space to store it compared to the disk space for a P-frame. The terms "size" and "bit size" may sometimes be used interchangeably.

[0036] The estimation of the number of bits for a GOP of a video stream to be transmitted may be performed by analyzing one or more already transmitted GOPs in the video stream. By analyzing multiple already transmitted GOPs, statistical trends based on the number of bits for each of the already transmitted GOPs may be used for a relatively accurate estimation of the number of bits for a GOP to be transmitted. Thus, the method may include statistically analyzing the number of bits for one or more preceding GOPs in the video stream. The statistical analysis of the number of bits for one or more preceding GOPs in the video stream may utilize a probabilistic approach in that a bit rate may be selected so that the risk of introducing latency is below a threshold probability. The threshold probability may preferably be relatively low. For example, when a camera captures a video sequence of a relatively static scene, P frames of a particular GOP in the video sequence are generally much smaller than I frames of the particular GOP. The statistical analysis may then estimate a suitable bit rate to be used in subsequent GOPs. The statistical analysis may further include determining trends in the scene captured by the camera. As an example, the trend may take into account an increasing rate of change of movement in a scene, which may result in larger P-frames, which may further optimize methods for smoothing the corresponding data traffic transmitted over the network.

[0037] Method 100 further includes, at 120, setting a latency requirement for the transmission of the video stream. As described above, the latency requirement is a requirement regarding the time difference between capturing / generating video frame data and transmitting the video frame data. The latency requirement may be a maximum latency that shall not be exceeded. The latency requirement may sometimes be referred to as latency. Thus, errors and potentially unnecessary latency caused by an estimated video bit rate over time to be transmitted may be prevented. The latency requirement may be set by a user. The latency requirement may depend on user experience, network capacity, the type of scene to be monitored, etc. As an example, a camera monitoring a gate and being used for real-time human communication may require near-real-time transmission of data, e.g., a 30 ms latency. Conversely, for a camera monitoring a parking lot, the latency may be significantly higher, e.g., 0.5 s. A target time at which the traffic shaper should transmit all data for one or more GOPs may further be calculated based on the frame rate and GOP length. The target time can therefore be seen as a timestamp at which subsequent I-frame transmissions occur in real time.

[0038] The method 100 further includes, at 130, determining an average minimum bitrate over the GOP based on the estimated number of bits and a time period corresponding to the time period represented by the duration of the GOP. The average minimum bitrate over the GOP may depend on the video resolution. That is, for example, if low latency is desired in a live stream, the average minimum bitrate over the GOP in the live stream may be relatively low, particularly for scenes captured by a live stream camera that are relatively visually static. The average minimum video bitrate over the GOP may be defined based on how much data needs to be transmitted during each frame period, such that all video data corresponding to the GOP is transmitted during the GOP period.

[0039] The method 100 further includes, for the video frames in the GOP, setting an output bit rate for transmission of the video frames based on the latency requirement and the average minimum video bit rate at 140, and transmitting the video frames using the output bit rate at 150, whereby transmission of each video frame in the GOP is considered individually. The output bit rate may be updated based on the estimated number of bits and a target time by which pushing the frame to be transmitted should be completed.

[0040] 2A-2C diagrammatically summarize the method 100. Here, a GOP 200 comprising one I-frame I1 and eight P-frames P1-P8 is illustrated, which should not be construed as limiting or confusing to those skilled in the art. That is, any suitable number of P-frames is possible within the scope of the claims. In the absence of latency, transmission of video frames may generally occur during any suitable time interval. Such a time interval may preferably be shorter than or equal to a frame rate-dependent time interval corresponding to the elapsed time between two subsequent captured frames, as qualitatively illustrated in FIGS. 2A-2C. If the time interval is significantly shorter than the frame rate-dependent time interval, the corresponding bit rate curve will have a narrow bit rate spike for each transmitted P-frame, despite the potentially insignificant size of each P-frame. Again, it can be seen that the I-frame is significantly larger than each of the subsequent P-frames. Therefore, this example suggests a relatively moderate change in the captured scene. A typical frame rate may be 30 frames per second (fps). The GOP length may be, for example, 30 frames or 60 frames. Thus, for a frame rate of 30 fps and a GOP length of 60 frames, the duration of the GOP is 2 seconds. As will be readily appreciated by those skilled in the art, other suitable frame rates and GOP lengths are possible within the scope of the claims.

[0041] FIG. 2A illustrates a typical transmission of data traffic for a GOP 200 over time, with relatively narrow spikes in the transmission bit rate during the transmission of I-frames. In this toy example, GOP 200 is assumed to have one I-frame I1 and a group of P-frames 210, which includes eight P-frames P1-P8. The apparent "area" of each frame in GOP 200 qualitatively indicates the relative bit size of the frames in the GOP. The frames shown in FIG. 2A(i) can be considered data packets. Thus, each frame I1, P1-P8 can be considered a data packet to be pushed across the network. Pushing data packets in this manner is equivalent to having minimal latency, since each individual data packet is transmitted as soon as the corresponding video frame is captured. Generally, and as can be seen in this example, I-frame I1 has a significantly larger bit size compared to the subsequent P-frames P1-P8, where P-frames P1-P8 are all assumed to have substantially similar bit sizes, although the bit sizes within a genuine group of P-frames may generally differ.

[0042] FIG. 2B illustrates a schematic diagram of smoothing a GOP 200 over time. The GOP 200 is smoothed according to the method 100 during transmission of the GOP 200. Here, a first latency requirement LR1 is set, defining the time interval during which bits of an I-frame should be pushed over the network. The I-frame I1 shown in FIG. 2A is split into two data packets and pushed over the network during a time interval corresponding to the pushing of two frames, e.g., the I-frame I1 and the first P-frame P1 of FIG. 2A. Therefore, the I-frame I1 should not be interpreted as being split into two different frames, but rather as being split into two different data packets. When the GOP 200 is transmitted over the network and played back on the receiver side, the corresponding video sequence is played back at normal speed and delayed by a time similar to the latency requirement. Smoothing requires that the remaining P frames P1-P8 of GOP 200 must be combined into larger packets in order for GOP 200 to be transmitted before the subsequent GOP begins. In this example, the final data packet may be the data packet comprising the seventh P7 and eighth P8 P frames, so that the GOP comprising data packets I1a, I1b, P1-P6, and larger P-frame packet PL1 comprising the seventh P7 and eighth P8 P-frames is pushed within target time TG1. Any pair of P-frames, such as the first P-frame P1 and second P-frame P2, may equally well be used to construct a similar data packet. This may result in a corresponding bitrate curve having a first bitrate peak 252 corresponding to the bitrate when pushing I-frame packets I1a and I1b over the network. Similarly, a second bitrate peak 254 may occur in bitrate curve 250 when pushing larger P-frame packet PL1 over the network. Generally, a data packet may have a size of about 1.5 kB. A relatively large I-frame may be split into thousands of data packets.Frames can be transmitted by such data packets in at least three ways: (i) one frame in one data packet, (ii) multiple frames in one data packet, or (iii) one frame in multiple data packets. In principle, it may even be possible to split a certain number of frames into another number of data packets. For example, two data packets may contain a total of three frames, with each data packet containing data corresponding to 1.5 frames.

[0043] FIG. 2C illustrates another example of smoothed data corresponding to GOP 200 over time, with data traffic being smoothed according to method 100 during transmission of GOP 200. Here, a second latency requirement LR2 is established, defining the time intervals during which I-frames should be pushed on the network. I-frame I1, shown in FIG. 2A, is now split into five data packets I1i-I1v of substantially equal bit size. P-frames P1-P8 are now combined into data packets each comprising two P-frames: a first P-frame packet comprising a first P1 and a second P2 P-frame, a second P-frame packet comprising a third P3 and a fourth P4 P-frame, and so on. This example may result in a corresponding bitrate curve 260 having a substantially constant bitrate when pushing I-frame packets I1i-I1v and subsequent P-frame packets. It should be appreciated that a larger time interval value of the latency requirement may result in a more constant bitrate curve when pushing data packets corresponding to bits of GOP 200, and vice versa. The bitrate curve may thereby deviate little or negligibly from the average minimum bitrate of the GOP. Therefore, the latency requirement may be viewed as a trade-off between the length of the time interval of the latency requirement and the smoothness of the corresponding bitrate curve. It should be appreciated that the example shown in FIG. 2C may be relatively ideal in terms of network load. Because the estimation of the number of bits for a GOP of a video stream to be transmitted may depend on GOPs that have already been transmitted, fluctuations may occur such that the average bitrate between two GOPs may deviate.

[0044] 2A-2C, the frame rate may appear to be similar to the rate at which data packets corresponding to frames in GOP 200 are transmitted. However, the data packets may equally well be transmitted at a lower rate, such that each data packet comprises multiple frames and each data packet is sent during a time period corresponding to the capture of the frames contained in the data packet.

[0045] The bit number estimation 110 may include buffering video data corresponding to the GOP to be transmitted and analyzing the buffered video data. Thus, all frames in the GOP may be buffered, and then the total bit size of all frames may be calculated, as well as the average bit rate required for the GOP to be transmitted during its target time. This may be followed by pushing data packets at a bit rate that matches the average bit rate of the GOP. In this situation, there may be a relatively large latency, but the bit rate may be further smoothed or even substantially constant while pushing the data packets corresponding to the GOP over the network.

[0046] Alternatively, the bit number estimation 110 may include determining the number of bits for an already encoded sequence of the video stream corresponding to the GOP. Thus, smoothing of already encoded video data may be possible. This may be preferable in the case of pre-buffering when an event, for example, a change in a scene captured by a video camera, is triggered, so that a data packet corresponding to a certain past time interval can be pushed. Such a certain past time interval may be, for example, the preceding 15 seconds, the preceding minute, etc.

[0047] The method 100 may further include updating an average minimum video bitrate over the remaining yet-to-be-transmitted frames of the GOP based on the actual number of bits of the already-transmitted frames of the GOP during transmission of the video frames. Thus, the average minimum video bitrate can be adjusted according to real-time conditions in the scene captured by the video camera. This embodiment may be preferred when the previous GOP experiences rapid changes in the captured scene and the current GOP experiences a relatively static captured scene.

[0048] With reference to FIG. 3, a computer-implemented method 300 for controlling the simultaneous transmission of multiple video streams is shown. Method 300 includes individually performing method 100 described above for each of the multiple video streams. Method 300 may include smoothing the video streams from multiple cameras, thereby causing the sum of the video streams' bitrate to be below a maximum allowable bitrate for transmission of data over the network. If the corresponding latency requirements of each video stream are relatively short, such that bitrate spikes occur for the multiple video streams, a second method may delay one or more video streams as necessary to cause the sum of the video streams' bitrate to be below the maximum allowable bitrate of the network.

[0049] The method 300 may further include individually setting a latency requirement for transmission of each of the multiple video streams, such that the latency requirement is different for each of the multiple video streams.

[0050] Method 300 may further include individually setting latency requirements for the transmission of each of the multiple video streams based on the type of each of the multiple video streams. The type of video stream may include a live stream to be stored for later review, etc. Playback of the live stream may experience a delay between capture and playback of a few seconds or less. If the type of video stream is a live stream, the latency requirement may thereby be a relatively short period of time. This may then set a constraint on the number of live streams being transmitted over the network. Method 300 may evaluate activity in each scene associated with each live stream, such that less active scenes are assigned, possibly temporarily, longer latency requirements. Alternatively, if I-frames of multiple live streams are transmitted substantially simultaneously, one or more live streams may be delayed to some extent relative to each other.

[0051] Instructions for implementing the methods described above are stored on a non-transitory computer-readable storage medium and executed on a device having processing capabilities.

[0052] With reference to Figure 4, a video stream formatter 400 for controlling the transmission of a video stream is shown very diagrammatically. The video stream formatter 400 may be implemented on any suitable device. By way of non-limiting example, the video stream formatter 400 may be implemented in a video camera, in particular a digital monitoring video camera. Other examples are fixed or portable computers, network gateways, etc. The features described above with respect to the method(s) 100, 300 also apply to the video stream formatter 400, when applicable. Therefore, to avoid excessive repetition, reference is made thereto.

[0053] The video stream formatter 400 comprises a circuit 420 .

[0054] The circuitry 420 is configured to provide overall control of the functions and operations of the video stream formatter 400. The circuitry 420 may include a processor, such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor is configured to execute program code stored in the circuitry 420 to perform the functions and operations of the video stream formatter 400.

[0055] Executable functions, described further below, may be stored in memory. The memory may be one or more of a buffer, flash memory, hard drive, removable media, volatile memory, non-volatile memory, random access memory (RAM), or other suitable device. In a typical arrangement, the memory may include non-volatile memory for long-term data storage and volatile memory that serves as system memory for the circuit 420. The memory may exchange data with the circuit 420 via a data bus. There may be associated control lines and an address bus between the memory and the circuit 420.

[0056] The functions and operations of the circuitry 420 may be embodied in the form of executable logic routines, such as computer code portions, software programs, etc., stored in a non-transitory computer-readable medium, such as a memory, of the video stream formatter 400 and executed by the circuitry 420, such as by using a processor. The functions and operations of the video stream formatter 400 may be standalone software applications or may form part of a software application that performs additional tasks related to the video stream formatter 400. The functions and operations described may take into account the manner in which a corresponding device is configured to perform. Also, while the functions and operations described may be implemented in software, such functions may also be performed via dedicated hardware or firmware, or some combination of hardware, firmware, and / or software.

[0057] The circuit 420 is configured to execute a data size estimation function 421 configured to estimate a number of bits for a group of pictures (GOP) of the video stream to be transmitted. The circuit 420 further comprises a latency setting function 422 configured to set a latency requirement for transmission of the video stream. The circuit 420 further comprises a bit rate determination function 423 configured to determine an average minimum video bit rate over the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP. The circuit 420 further comprises an output bit rate setting function configured to set an output bit rate for transmission of the video frames based on the latency requirement and the average minimum bit rate for the video frames in the GOP. The circuit 420 further comprises a transmission function 425 configured to transmit the video frames using the output bit rate.

[0058] The data size estimation function 421 may further be configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

[0059] The data size estimation function 421 may be further configured to buffer video data corresponding to a GOP to be transmitted and analyze the buffered video data.

[0060] The data size estimation function 421 may further be configured to determine the number of bits for an already encoded sequence of the video stream corresponding to the GOP.

[0061] The circuit 420 may further comprise a bitrate update function configured to update the average minimum video bitrate over the remaining yet to be transmitted frames of the GOP based on the actual number of bits of the already transmitted frames of the GOP.

[0062] Circuitry 420 may be further configured to perform main function 426 configured to individually perform functions 421-425 for each of a plurality of video streams, as described above.

[0063] Those skilled in the art will appreciate that the present invention is by no means limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

[0064] For example, a particular video stream shaper 400 may control the transmission of two independent video streams. In such a situation, the latency requirements of one of the two video streams may take into account the properties of the remaining video stream to optimize smoothing of the transmitted data. Those skilled in the art will appreciate that this may also apply to three or more independent video streams.

[0065] Additional variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

Claims

1. A computer-implemented method (100) for controlling transmission of a video stream, said method (100) comprising: estimating (110) a number of bits for a group of pictures (GOP) (200) of the video stream to be transmitted, the GOP comprising an intra frame and one or more inter frames; Setting (120) a latency requirement for the transmission of the video stream, the latency requirement being a requirement regarding a time difference between capturing / generating video frame data and transmitting the video frame data; determining (130) an average minimum video bitrate over the GOP (200) based on the estimated number of bits and a time corresponding to a time period represented by a duration of the GOP (200); For each video frame in the GOP (200), setting (140) an output bitrate for transmission of the video frame based on the latency requirement and the average minimum video bitrate by determining a number of data packets over which the video frame should be spread or determining that the video frame should be transmitted in the same data packet as another video frame of the GOP; transmitting (150) the video frames using the output bit rate; A method (100) comprising:

2. 2. The method of claim 1, wherein estimating the number of bits comprises statistically analyzing the number of bits for one or more preceding GOPs in the video stream.

3. 2. The method of claim 1, wherein estimating the number of bits comprises buffering video data corresponding to the GOP to be transmitted and analyzing the buffered video data.

4. 2. The method of claim 1, wherein estimating the number of bits comprises determining a number of bits for an already-encoded sequence of the video stream that corresponds to the GOP.

5. 2. The method of claim 1, further comprising: during transmission of the video frames in the GOP, updating the average minimum video bit rate over remaining yet-to-be-transmitted frames of the GOP based on an actual number of bits in already-transmitted frames of the GOP.

6. 10. A computer-implemented method (300) for controlling simultaneous transmission of multiple video streams, the method (300) comprising individually performing the method of claim 1 for each of the multiple video streams.

7. 7. The method of claim 6, further comprising individually setting a latency requirement for transmission of each of the plurality of video streams, such that the latency requirement is different for each of the plurality of video streams.

8. 7. The method of claim 6, further comprising individually setting the latency requirement for the transmission of each of the plurality of video streams based on a video stream type of each of the plurality of video streams.

9. A non-transitory computer-readable storage medium storing instructions for implementing the method of any one of claims 1 to 8 when executed on a device having processing capabilities.

10. A video stream formatter (400) for controlling the transmission of a video stream, said video stream formatter (400) comprising: a data size estimation function (421) configured to estimate the number of bits for a group of pictures (GOP) (200) of the video stream to be transmitted, the GOP comprising an intra frame and one or more inter frames; a latency setting function (422) configured to set a latency requirement for the transmission of the video stream, the latency requirement being a requirement regarding a time difference between capturing / generating video frame data and transmitting the video frame data; a bitrate determination function (423) configured to determine an average minimum video bitrate over the GOP (200) based on the estimated number of bits and a time period corresponding to a time period represented by the duration of the GOP (200); an output bitrate setting function (424) configured to set, for each video frame in the GOP (200), an output bitrate for transmission of the video frame based on the latency requirement and an average minimum bitrate by determining the number of data packets over which the video frame should be spread or by determining that the video frame should be transmitted in the same data packet as another video frame of the GOP; a transmitting function (425) configured to transmit the video frames using the output bit rate; A video stream formatter (400) comprising a circuit (420) configured to perform:

11. 11. The video stream formatter of claim 10, wherein the data size estimation function is further configured to statistically analyze the number of bits for one or more preceding GOPs in the video stream.

12. 11. The video stream formatter (400) of claim 10, wherein the data size estimation function (421) is further configured to buffer video data corresponding to the GOP (200) to be transmitted and analyze the buffered video data.

13. 11. The video stream formatter (400) of claim 10, wherein the data size estimation function (421) is further configured to determine a number of bits for an already encoded sequence of the video stream corresponding to the GOP (200).

14. 14. The video stream formatter (400) of claim 10, further comprising a bitrate update function configured to update the average minimum video bitrate over the remaining yet-to-be-transmitted frames of the GOP (200) based on the actual number of bits in already-transmitted frames of the GOP (200).

Images