Method and apparatus for controlling transmission of video streams

Abstract

A method and apparatus for controlling the transmission of a video stream. A computer-implemented method for controlling the transmission of a video stream is provided. The method includes: estimating the number of bits of a group of pictures (GOP) of a video stream to be transmitted; setting a delay requirement for transmission of the video stream; determining an average minimum video bitrate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP; for video frames in the GOP, setting an output bitrate for transmission of the video frames based on the delay requirement and the average minimum video bitrate; and transmitting the video frames using the output bitrate.

Description

Technical Field

[0001] The present disclosure relates to methods and apparatus for controlling the transmission of video streams. Background Art

[0002] In video coding, a group of pictures, or GOP, structure specifies the order in which intra and inter frames are arranged. A GOP is a collection of consecutive pictures in the encoded video stream. Each encoded video stream consists of consecutive GOPs, from which visible frames are generated. Encountering a new GOP in a compressed video stream means that the decoder does not need any previous frames to decode the next frame, allowing for fast seeking within the video. A GOP begins with an intra frame and ends with the last frame in the group, preceding the next intra frame in the encoded video stream. Intra frames are typically significantly larger than inter frames, often 10 to 50 times larger. Consequently, transmitting an encoded video stream over a network results in a relatively high instantaneous bitrate when transmitting intra frames, and a relatively low bitrate when transmitting subsequent inter frames. Consequently, bitrate spikes often occur when transmitting intra frames. Modern network and internet connections can generally handle these bitrate spikes without further compressing the video stream. However, if multiple cameras are installed on a given network, and multiple bit rate spikes occur simultaneously, the total data transmission bit rate may momentarily exceed the network's maximum data transmission bit rate, even if there is sufficient bandwidth for multiple cameras on average. This can result in data packet loss, video quality degradation, and other issues. Therefore, it is necessary to prevent such transmission bit rate overloads in the network. Summary of the Invention

[0003] Therefore, it is an object of the present invention to provide a method for controlling the transmission of a video stream in order to prevent an overload of the transmission bit rate in the network.

[0004] According to a first aspect, a computer-implemented method for controlling the transmission of a video stream is provided. The method includes estimating a number of bits for a group of pictures (GOP) of the video stream to be transmitted. The method further includes setting a delay requirement for the transmission of the video stream. The method further includes determining an average minimum video bitrate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP. The method further includes: for video frames in the GOP, setting an output bitrate for transmission of the video frames based on the delay requirement and the average minimum video bitrate; and transmitting the video frames using the output bitrate.

[0005] Traditionally, surveillance networks are closed systems where cables are permanently installed and transmission capacity is guaranteed (this guarantee stems, among other things, from predictable bit rate profiles). This approach allows data from surveillance cameras to be transmitted over a network that can also carry other types of traffic. This may be more suitable for long-distance transmission of video data over links where transmission capacity cannot be guaranteed.

[0006] Compared to conventional data traffic shapers in which the maximum bit rate is set and fixed, the present disclosure provides a dynamic data traffic smoother that reduces the load on the network transmitting the data. Furthermore, due to the possibility of reducing the load on the network, the data transmission speed between other devices in the network can be improved.

[0007] In this document, bitrate refers to the number of bits transmitted per second over a digital network. A bitrate curve refers to the bitrate as a function of time. A bitrate peak in a bitrate curve refers to a relatively short time interval of the bitrate curve during which the bitrate is significantly greater than the average bitrate, for example, a time interval during which an intraframe is transmitted. Unless otherwise explicitly stated, the network referred to is any type of data transmission network (possibly wireless) over which the transmission of the video frames in question occurs. The network can be any type of network used to transmit digital information, such as, for example, a local area network, a public network, etc. The terms "transmit" and "push" data over a network are used interchangeably throughout this document.

[0008] The average minimum video bit rate across a GOP is defined as the amount of data that needs to be transmitted during each frame period so that all video data corresponding to the GOP will be transmitted during the GOP period. Given a GOP length of 30 frames captured at a frame rate of 30 frames per second, for example, the frame period can be approximately 30-40 milliseconds.

[0009] This method can facilitate a smooth bitrate profile for a GOP. In this context, a smooth bitrate profile refers to a substantially constant transmission bitrate, which is the average bitrate across the GOP. This prevents the maximum bitrate of the network over which the video stream is transmitted from being temporarily exceeded. Consequently, the risk of information loss in, for example, live streaming can be reduced.

[0010] In this specification, latency refers to a time interval where the start of the time interval is the timestamp at which the capture or generation of a video frame is complete, and the end of the time interval is the timestamp at which the video frame has been transmitted. Thus, a latency requirement can refer to the maximum acceptable time interval. The method can facilitate shorter latency without compromising video quality, which can be advantageous in, for example, live streaming.

[0011] The estimation of the number of bits can include statistically analyzing the number of bits of one or more preceding GOPs in the video stream. This can facilitate an appropriate estimation of the number of bits of the GOP to be transmitted. This can further facilitate the smooth dynamic nature of the data service, as the instantaneous minimum bit rate required to transmit the upcoming GOP is being continuously monitored and updated.

[0012] The estimation of the number of bits may further include buffering video data corresponding to the GOP to be transmitted, and analyzing the buffered video data.

[0013] The bit count estimation can further include determining the bit count of the encoded sequence of the video stream corresponding to the GOP. Thus, the encoded sequence of the video stream can be smoothed and transmitted. Assuming that shorter delays are less important, this can be advantageous because a substantially accurate average bit rate can be calculated from the captured and stored GOPs. This can further provide a more refined estimate of the bit count of the GOP.

[0014] The method may further comprise: updating the average minimum video bit rate of the remaining frames in the GOP that have not yet been transmitted based on the actual number of bits of the frames in the GOP that have already been transmitted during the transmission of the video frames in the GOP. Thus, the average minimum video bit rate can be updated according to the real-time environment in the scene captured by the camera. Therefore, dynamic updating of the average minimum bit rate can be performed during the transmission of the video frames in the GOP, which can further contribute to avoiding network overload. This may be more preferable if the GOP time length (i.e., the time interval for transmitting the GOP) is relatively long. Alternatively or additionally, if the transmission bit rate of the previous GOP is subject to rapidly changing capture scenes, while the current GOP is subject to relatively static capture scenes, this can reduce the average minimum video bit rate.

[0015] According to a second aspect, there is provided a computer-implemented method for controlling simultaneous transmission of a plurality of video streams, the method comprising individually performing the method according to the above features of the first aspect for each of the plurality of video streams.

[0016] If multiple video streams' intraframes are transmitted simultaneously, network overload may occur due to the resulting increase in multiple bitrate peaks. This approach can help smooth the data traffic being transmitted, thereby avoiding such network overload. This can prevent packet loss or video resolution degradation, for example.

[0017] The method according to the second aspect may further include setting a delay requirement for transmission of each of the plurality of video streams individually, such that the delay requirement is different for each of the plurality of video streams. This may be advantageous if different types of cameras are used among the plurality of cameras, and / or if different resolutions, GOP lengths, etc. are used among a plurality of cameras of the same type. This may further enhance the dynamic nature of the method.

[0018] The method according to the second aspect may further include: separately setting a delay requirement for the transmission of each of the multiple video streams based on the type of the video stream of each of the multiple video streams. The type of the video stream may be a live stream or a non-live stream, etc. Different video resolutions may be suitable for different types of video streams. For example, a non-live stream may have a larger video resolution than a live stream. In addition to the higher video resolution, the separate setting of the delay requirement may allow the transmission bit rate of the non-live stream to be set to be substantially constant, which may prevent data loss. Therefore, the delay of the non-live stream may be much greater than that of the live stream. In addition, the bandwidth of the network may be utilized more efficiently, thereby allowing a larger number of such non-live streams to be stacked for simultaneous transmission over the network.

[0019] According to a third aspect, there is provided a non-transitory computer-readable storage medium having stored thereon instructions for implementing the method according to the first and / or second aspect when the instructions are executed on a device having processing capabilities.

[0020] Where applicable, the features and advantages described above in relation to the first and second aspects also apply to this third aspect. To avoid unnecessary repetitions, reference is made to the above.

[0021] According to a fourth aspect, there is provided a video stream shaper for controlling transmission of a video stream, the video stream shaper comprising circuitry configured to:

[0022] a data size estimation function configured to estimate the number of bits of a group of pictures GOP of a video stream to be transmitted,

[0023] A delay setting function configured to set a delay requirement for transmission of a video stream,

[0024] a bitrate determination function configured to determine an average minimum video bitrate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP,

[0025] an output bitrate setting function configured to set an output bitrate for transmission of video frames based on a delay requirement and an average minimum bitrate for video frames in a GOP, and

[0026] The transmit function is configured to transmit video frames using the output bitrate.

[0027] Where applicable, the features and advantages described above in relation to the first aspect also apply to this second aspect. To avoid unnecessary repetitions, reference is made to the above.

[0028] The data size estimation function may further be configured to statistically analyze the number of bits of one or more preceding GOPs in the video stream.

[0029] The data size estimation function may be further configured to buffer video data corresponding to the GOP to be transmitted, and analyze the buffered video data.

[0030] The data size estimation function may further be configured to determine the number of bits of an already encoded sequence of the video stream corresponding to a GOP.

[0031] The circuit may further comprise a bit rate update function configured to update an average minimum video bit rate of remaining frames of the GOP that have not yet been transmitted based on actual number of bits of frames of the GOP that have already been transmitted.

[0032] The circuit may be further configured to perform a main function configured to individually perform the above-mentioned functions of the video stream shaper for each of the plurality of video streams.

[0033] Where applicable, the above features of the method also apply to this second aspect. To avoid unnecessary repetitions, reference is made to the above.

[0034] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "an / the [element, device, component, means, step, etc.]" should be interpreted as referring to at least one instance of the element, device, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and additional objects, features and advantages of the present invention will be better understood through the following illustrative and non-limiting detailed description of preferred embodiments with reference to the accompanying drawings, in which like reference numerals are used for similar elements, and in which:

[0036] Figure 1 A flow chart of a method for controlling the transmission of a video stream is schematically shown.

[0037] Figures 2A to 2C The transmission of data traffic of a group of pictures GOP over time is schematically shown.

[0038] Figure 3 A method for controlling the transmission of multiple video streams is schematically illustrated.

[0039] Figure 4 A video stream shaper for controlling the transmission of a video stream is shown highly schematically. DETAILED DESCRIPTION

[0040] The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which presently preferred embodiments of the invention are illustrated. However, the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness and to fully convey the scope of the invention to those skilled in the art.

[0041] Combine Figure 1 , which shows a flow chart of a computer-implemented method 100 for controlling the transmission of a video stream. The method 100 can be implemented on any suitable device. According to a non-limiting example, the method 100 can be implemented in a camera, in particular in a digital surveillance camera. Other examples are fixed or portable computers, gateways, etc. The method can be regarded as a so-called traffic shaper for data traffic transmitted over a network. The method 100 comprises estimating 110 the number of bits of a group of pictures GOP of a video stream to be transmitted. A GOP may comprise different picture types:

[0042] Intra-coded pictures (I-frames) are pictures that are coded independently of all other pictures. In decoding order, each GOP starts with an I-frame. I-frames can also be called key frames.

[0043] A predictive coded picture (P-frame) is a picture that contains motion compensated difference information relative to a previously decoded picture (either a P-frame or an I-frame). P-frames can be created through pixel-level analysis, block-level analysis, or vector analysis.

[0044] A bi-predictively coded picture (B-frame) is a picture that contains motion-compensated difference information relative to the previously decoded picture (I-frame or P-frame) and the picture following the B-frame (I-frame of a P-frame).

[0045] P frames and B frames are often collectively referred to as inter-frames.

[0046] The number of bits in an intra frame is typically significantly greater than the number of bits in an inter frame following the intra frame, as the inter frame typically only includes differences relative to the intra frame. However, this may not be the case when capturing scenes containing significant motion. Thus, in camera surveillance, for example, during nighttime (where there may be little or no change in the scene captured by the camera during a particular time interval), the file size of the inter frames in the captured video stream sequence may be essentially negligible. Furthermore, in such situations, a relatively large GOP length and / or a low frame rate may be used, where the GOP length refers to the number of frames between two consecutive intra frames. However, in order to adequately capture rapid changes in the scene captured by the camera in camera surveillance, a relatively short GOP length may be preferred, regardless of the current (potentially low) activity in the scene being captured. Throughout this disclosure, discussion of pictures in a GOP refers only to I-frames and P-frames. Those skilled in the art will recognize that other types of frames (i.e., B-frames) may also be considered within the scope of the claims. Furthermore, throughout this disclosure, static GOP lengths are contemplated, but those skilled in the art will appreciate that dynamic GOP lengths can be implemented to, for example, conserve bandwidth and facilitate data traffic on a network. When describing / discussing the size of a video frame (I-frame or P-frame), it will be understood that reference is made to the corresponding disk space required to store such a video frame. Thus, it will be understood that an I-frame being larger than a P-frame is equivalent to requiring more disk space to store the I-frame than a P-frame. The terms "size" and "bit size" are sometimes used interchangeably.

[0047] Estimating the number of bits in a GOP of a video stream to be transmitted can be performed by analyzing one or more already transmitted GOPs in the video stream. By analyzing multiple already transmitted GOPs, statistical trends in the number of bits in each of the already transmitted GOPs can be used to relatively accurately estimate the number of bits in the GOP to be transmitted. Therefore, the method can include statistically analyzing the number of bits in one or more preceding GOPs in the video stream. The statistical analysis of the number of bits in one or more preceding GOPs in the video stream can utilize a stochastic approach, as the bit rate can be selected such that the risk of introducing delays is below a threshold probability. The threshold probability can preferably be relatively low. For example, if a camera captures a video sequence of a relatively static scene, the P-frames of a particular GOP in the video sequence are typically much smaller than the I-frames of that particular GOP. The statistical analysis can then estimate an appropriate bit rate to be used in subsequent GOPs. The statistical analysis can further include determining trends in the scene captured by the camera. For example, the trend can indicate an increase in the rate of change of motion in the scene, which results in larger P-frames. This can further optimize the method for smoothing the corresponding data traffic transmitted over the network.

[0048] Method 100 further includes setting 120 a delay requirement for the transmission of the video stream. As described above, the delay requirement is a requirement for the time difference between capturing / generating video frame data and transmitting it. The delay requirement can be a maximum delay that should not be exceeded. The delay requirement can sometimes be indicated as a delay. This can prevent errors caused by estimating the video bit rate across the GOP to be transmitted and potentially induce undesirable delays that accumulate and increase over time. The delay requirement can be set by the user. The delay requirement may depend on user experience, network capacity, or the type of scene to be monitored. For example, a camera monitoring a gate and used for real-time personnel communication may require near-real-time data transmission, such as a 30 millisecond delay. In contrast, a camera monitoring a parking lot may require significantly higher delays, such as 0.5 seconds. The target time by which the traffic shaper should transmit all data for one or more GOPs can be further calculated based on the frame rate and GOP length. Therefore, the target time can be considered a timestamp within which the transmission of subsequent I-frames is completed in real time.

[0049] The method 100 further includes determining 130 an average minimum bit rate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP. The average minimum bit rate across the GOP can depend on the video resolution. That is, assuming a shorter delay is desired, such as in a live stream, the average minimum bit rate across the GOP in the live stream can be relatively low (especially for relatively visually static scenes captured by a live stream camera). The average minimum video bit rate over the GOP can be defined based on how much data needs to be transmitted during each frame period, so that all video data corresponding to the GOP will be transmitted during the GOP period.

[0050] The method 100 further includes, for the video frames in the GOP, setting 140 an output bit rate for transmitting the video frames based on the delay requirement and the average minimum video bit rate, and transmitting 150 the video frames using the output bit rate. Here, the transmission of each video frame in the GOP is therefore considered separately. The output bit rate can be updated based on the estimated number of bits and the target time by which the push of the frame to be transmitted should have been completed.

[0051] Figures 2A to 2CThe method 100 is summarized graphically. In this document, the GOP 200 is illustrated as including one I-frame I1 and eight P-frames P1-P8, which should not be construed as limiting or confusing those skilled in the art. That is, any sufficient number of P-frames is possible within the scope of the claims. The transmission of the video frames can generally be completed during any appropriate time interval without any delay. Preferably, such a time interval can be shorter than or equal to the frame rate of the time interval corresponding to the time elapsed between two consecutive captured frames, which depends on the frequency, such as Figures 2A to 2C Qualitatively shown. If the time interval is significantly shorter than the time interval that depends on the frame rate, the corresponding bit rate curve will have a narrower bit rate peak for each transmitted P frame, although the size of the individual P frames may not be noticeable. Similarly, it can be seen that the I frame is significantly larger than each subsequent P frame. Therefore, this example shows that the changes in the captured scene are relatively mild. A typical frame rate may be 30 frames per second (fps). The GOP length may be, for example, 30 frames or 60 frames. Therefore, for a frame rate of 30 fps and a GOP length of 60 frames, the duration of the GOP is two seconds. As will be readily appreciated by those skilled in the art, other suitable frame rates and GOP lengths are also possible within the scope of the claims.

[0052] exist Figure 2A In FIG, a typical transmission of data traffic of a GOP 200 is schematically shown over time, wherein the transmission bit rate exhibits a relatively narrow peak during the transmission of an I frame. In this small example, it is assumed that the GOP 200 has one I frame I1 and a set of eight P frames 210, P frames P1-P8. The visible "area" of the individual frames of the GOP 200 qualitatively indicates the relative bit size of the frames in the GOP. Figure 2A The frames shown in (i) can be considered as data packets. Thus, each frame I1; P1-P8 can be considered as a data packet pushed through the network. Pushing the data packets in this way amounts to having minimal latency because each individual data packet is transmitted as soon as the corresponding video frame is captured. Overall, it can also be seen in this example that the I frame I1 has a significantly larger bit size than the subsequent P frames P1-P8, where the P frames P1-P8 are all assumed to have a substantially similar bit size, but the bit size within a set of real P frames can generally be different.

[0053] exist Figure 2B In FIG, a smoothing of a GOP 200 over time is schematically shown. When transmitting the GOP 200, the GOP 200 is smoothed according to the method 100. Here, a first delay requirement LR1 is set, defining the time interval of the bits of an I frame to be pushed through the network. Figure 2AThe I frame I1 shown in FIG is divided into two data packets in this document and is transmitted in parallel with the push of two frames (e.g., Figure 2A The GOP 200 is pushed over the network during the time interval corresponding to the first I-frame I1 and the first P-frame P1. Therefore, the I-frame I1 should not be interpreted as being divided into two different frames, but rather as being divided into two different data packets. When the GOP 200 has been transmitted over the network and played at the receiver, the corresponding video sequence is played at normal speed and delayed by a time similar to the delay requirement. Smoothness requires that the remaining P-frames P1-P8 of the GOP 200 must be bundled into a larger packet so that the GOP 200 can be transmitted before the subsequent GOP begins. In this example, for the GOP to be pushed within the target time TG1 (herein, including data packets I1a, I1b, P1-P6 and the larger P-frame packet PL1 containing the seventh P-frame P7 and the eighth P-frame P8), the last data packet can be the data packet containing the seventh P-frame P7 and the eighth P-frame P8. Any pair of P-frames (e.g., the first P-frame P1 and the second P-frame P2, etc.) can also be used to form a similar data packet. This can result in a corresponding bit rate curve having a first bit rate peak 252, which corresponds to the bit rate when pushing I frame packets I1a and I1b through the network. Similarly, when pushing larger P frame packets PL1 through the network, a second bit rate peak 254 can occur in bit rate curve 250. Typically, a data packet can have a size of about 1.5kB. Relatively large I frames can be divided into thousands of data packets. Frames can be transmitted in at least three ways via these data packets: (i) one frame in one data packet, (ii) multiple frames in one data packet, or (iii) one frame in multiple data packets. In principle, a specific number of frames can be further divided into another number of data packets. For example, two data packets can contain a total of three frames, where each data packet contains data corresponding to 1.5 frames.

[0054] exist Figure 2C In FIG, another example of smoothing the data corresponding to GOP 200 in time of the data traffic to be smoothed according to method 100 is schematically shown when transmitting GOP 200. Here, a second delay requirement LR2 is set, which defines the time interval in which the I frame will be pushed through the network. Figure 2AThe I-frame I1 shown in FIG is divided into five data packets I1i-I1v of substantially equal bit size herein. The P-frames P1-P8 are bundled here in data packets, each data packet comprising two P-frames, i.e. a first P-frame packet comprising a first P-frame P1 and a second P-frame P2, a second P-frame packet comprising a third P-frame P3 and a fourth P-frame P4, and so on. When pushing the I-frame packets I1i-I1v and subsequent P-frame packets, this example may result in a corresponding bit rate curve 260 having a substantially constant bit rate. It should be understood that when pushing data packets corresponding to bits of a GOP 200, a larger time interval value of the delay requirement may result in a more constant bit rate curve, and vice versa. The bit rate curve may therefore deviate only slightly or negligibly from the average minimum bit rate of the GOP. Therefore, the delay requirement may be regarded as a compromise between the length of the time interval of the delay requirement and the smoothness of the corresponding bit rate curve. It should be understood that in Figure 2C The example shown in may be relatively ideal for network load. Since the estimate of the number of bits of a GOP of the video stream to be transmitted may depend on the GOP that has already been transmitted, it may change, so that the average bit rate between two GOPs may deviate.

[0055] exist Figures 2A to 2C , the frame rate may appear to be approximately the rate at which data packets corresponding to frames in GOP 200 are transmitted. However, data packets may also be transmitted at a lower rate such that each data packet includes multiple frames, wherein each data packet is sent during a time period corresponding to the capture of the frames contained in the data packet.

[0056] The estimation 110 of the number of bits can include buffering the video data corresponding to the GOP to be transmitted and analyzing the buffered video data. Thus, all frames in the GOP can be buffered, and then the total bit size of all frames and the average bit rate required for transmitting the GOP during its target time can be calculated. Thereafter, the data packets are pushed at a bit rate that matches the average bit rate of the GOP. In this case, although the bit rate can be further smoothed (or even substantially constant) when pushing the data packets corresponding to the GOP through the network, there may be a relatively large delay.

[0057] Alternatively, estimating the number of bits 110 may comprise determining the number of bits of the already encoded sequence of the video stream corresponding to the GOP. Thus, smoothing of the already encoded video data is possible. This may be advantageous in the case of pre-caching when an event is triggered (e.g., a change in the scene captured by the camera), making it possible to push data packets corresponding to a specific elapsed time interval. Such a specific elapsed time interval may be, for example, the last 15 seconds or the last minute, etc.

[0058] Method 100 may further include, during transmission of video frames in a GOP, updating an average minimum video bitrate for the remaining, yet-to-be-transmitted frames across the GOP based on the actual number of bits of the frames of the GOP that have already been transmitted. Thus, the average minimum video bitrate may be adjusted based on real-time conditions in the scene being captured by the camera. This embodiment may be preferred if the previous GOP experienced a rapidly changing captured scene, while the current GOP experienced a relatively static captured scene.

[0059] Combine Figure 3 , which illustrates a computer-implemented method 300 for controlling the simultaneous transmission of multiple video streams. Method 300 includes performing the above-described method 100 separately for each of the multiple video streams. Method 300 may thus include smoothing the video streams from the multiple cameras so that the sum of the video streams has a bit rate below the maximum permissible bit rate for data transmission over the network. If the respective delay requirements of the individual video streams are relatively short, resulting in bit rate spikes across the multiple video streams, a second method may delay one or more of the video streams, if necessary, so that the sum of the multiple video streams has a bit rate below the maximum permissible bit rate for the network.

[0060] The method 300 may further include individually setting a delay requirement for transmission of each of the plurality of video streams, such that the delay requirement is different for each of the plurality of video streams.

[0061] The method 300 may further include setting a delay requirement for the transmission of each of the multiple video streams individually based on the type of video stream for each of the multiple video streams. The type of video stream may include a live stream that is stored for later viewing, etc. The playback of the live stream may experience a delay of several seconds or less between capture and playback. If the type of video stream is a live stream, the delay requirement may be a relatively short time period. This may then place constraints on the number of live streams transmitted over the network. The method 300 may estimate the activity in the corresponding scenes associated with each live stream, so that less active scenes are assigned a temporary, possibly longer, delay requirement. Alternatively, if the I frames of multiple live streams are transmitted substantially simultaneously, one or more live streams may be delayed to some extent relative to each other.

[0062] Instructions for implementing the above methods are stored on a non-transitory computer-readable storage medium and executed on a device with processing capabilities.

[0063] Combine Figure 4, which highly schematically illustrates a video stream shaper 400 for controlling the transmission of a video stream. Video stream shaper 400 can be implemented on any suitable device. According to a non-limiting example, video stream shaper 400 can be implemented in a camera, in particular a digital surveillance camera. Other examples include a fixed or portable computer, a network gateway, etc. The features described above in connection with methods 100 and 300 also apply to video stream shaper 400. To avoid unnecessary repetition, reference is made to the above text.

[0064] Video stream shaper 400 includes circuit 420 .

[0065] The circuit 420 is configured to perform overall control of the functions and operations of the video stream shaper 400. The circuit 420 may include a processor such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor is configured to execute program code stored in the circuit 420 to perform the functions and operations of the video stream shaper 400.

[0066] The executable functions described further below may be stored in the memory. The memory may be one or more of a cache, flash memory, a hard drive, removable media, volatile memory, non-volatile memory, random access memory, RAM, or other suitable devices. In a typical arrangement, the memory may include non-volatile memory for long-term data storage and volatile memory used as system memory for circuit 420. The memory may exchange data with circuit 420 via a data bus. Accompanying control lines and an address bus may exist between the memory and circuit 420.

[0067] The functions and operations of the circuit 420 may be implemented in the form of executable logic routines (e.g., computer code portions, software programs, etc.) stored on a non-transitory computer-readable medium (e.g., memory) of the video stream shaper 400 and executed by the circuit 420, for example, using a processor. The functions and operations of the video stream shaper 400 may be stand-alone software applications or form part of a software application that performs additional tasks associated with the video stream shaper 400. The functions and operations described may be considered methods that the corresponding apparatus may be configured to perform. Furthermore, while the functions and operations described may be implemented in software, such functions may also be implemented via dedicated hardware or firmware, or some combination of hardware, firmware, and / or software.

[0068] Circuitry 420 is configured to execute a data size estimation function 421 configured to estimate the number of bits of a group of pictures (GOP) of a video stream to be transmitted. Circuitry 420 further includes a delay setting function 422 configured to set a delay requirement for transmission of the video stream. Circuitry 420 further includes a bitrate determination function 423 configured to determine an average minimum video bitrate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by the duration of the GOP. Circuitry 420 further includes an output bitrate setting function 424 configured to set an output bitrate for transmission of the video frames in the GOP based on the delay requirement and the average minimum bitrate. Circuitry 420 further includes a transmission function 425 configured to transmit the video frames using the output bitrate.

[0069] The data size estimation function 421 may be further configured to statistically analyze the number of bits of one or more preceding GOPs in the video stream.

[0070] The data size estimation function 421 may be further configured to buffer video data corresponding to the GOP to be transmitted, and analyze the buffered video data.

[0071] The data size estimation function 421 may be further configured to determine the number of bits of an already encoded sequence of the video stream corresponding to a GOP.

[0072] The circuit 420 may further include a bit rate update function configured to update the average minimum video bit rate of the remaining frames of the GOP that have not yet been transmitted based on the actual number of bits of the frames of the GOP that have already been transmitted.

[0073] The circuit 420 may be further configured to execute a main function 426 configured to individually execute the above-described functions 421 to 425 for each of the plurality of video streams.

[0074] It will be appreciated by those skilled in the art that the present invention is not limited to the preferred embodiments described above. On the contrary, numerous modifications and variations are possible within the scope of the appended claims.

[0075] For example, a particular video stream shaper 400 may control the transmission of two independent video streams. In this case, the delay requirements of one of the two video streams may take into account the properties of the remaining video stream to optimize the smoothness of the transmitted data. Those skilled in the art will appreciate that this can also be applied to more than two independent video streams.

[0076] Additionally, variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

Claims

1. A computer-implemented method for controlling transmission of a video stream, the method comprising: estimating the number of bits of a group of pictures (GOP) of the video stream to be transmitted, the GOP comprising an intra frame and one or more inter frames; setting a delay requirement for the transmission of the video stream, wherein the delay requirement is a requirement on a time difference between capturing / generating video frame data for the intra-frame and transmitting the video frame data for the intra-frame; determining an average minimum video bit rate across the GOP based on the estimated number of bits and a time corresponding to a time period represented by a duration of the GOP; for the intra frames in the GOP, setting an output bit rate for transmission of the intra frames based on the delay requirement and the average minimum video bit rate by determining a number of data packets into which the intra frames are to be dispersed; transmitting the intra-frame using the output bit rate; For an inter-frame in the GOP, determining based on the average minimum video bit rate whether the inter-frame is to be transmitted in the same data packet as another inter-frame in the GOP or as a separate data packet; and The data packet including the inter-frame is transmitted.

2. The method according to claim 1, wherein Estimating the number of bits includes statistically analyzing the number of bits of one or more previous GOPs in the video stream.

3. The method according to claim 1, wherein Estimating the number of bits includes buffering video data corresponding to the GOP to be transmitted, and analyzing the buffered video data.

4. The method according to claim 1, wherein Estimating the number of bits includes determining the number of bits of an already encoded sequence of the video stream corresponding to the GOP.

5. The method according to claim 1, further comprising: During transmission of the video frames in the GOP, the average minimum video bit rate of the remaining frames of the GOP that have not yet been transmitted is updated based on the actual number of bits of the frames of the GOP that have already been transmitted.

6. A computer-implemented method for controlling the simultaneous transmission of a plurality of video streams, the method comprising separately performing the method of claim 1 for each of the plurality of video streams.

7. The method according to claim 6, further comprising: The delay requirement is set individually for the transmission of each of the plurality of video streams, such that the delay requirement is different for each of the plurality of video streams.

8. The method according to claim 6, further comprising: The delay requirement is individually set for the transmission of each of the plurality of video streams based on a type of the video stream of each of the plurality of video streams.

9. A non-transitory computer-readable storage medium having instructions stored thereon for implementing the method of claim 1 when the instructions are executed on a device having processing capabilities.

10. A video stream shaper for controlling transmission of a video stream, the video stream shaper comprising a circuit, the circuit comprising a processor and a memory, the processor being configured to execute program code stored in the memory to perform the method according to any one of claims 1 to 5.

Images