Method for controlling the bit rate of a video stream captured with an image capture device

Abstract

To provide a method, a storage medium, and a control unit that control the bitrate of a video stream captured using an image acquisition device.SOLUTION: In response to determination of a current bitrate value of a video stream captured using an image acquisition device and detection that the current bitrate value exceeds a threshold, at least one parameter of an edge filter is adjusted to further reduce the amount of edges in at least one area of the video stream, thereby controlling the bitrate.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates generally to the field of camera surveillance, and more particularly to a method and control unit for controlling the bit rate of a video stream captured by an image capture device. [Background technology]

[0002] In various camera surveillance applications, it is sometimes necessary to mask objects in the video stream captured by the camera. Some important reasons for masking objects are to preserve privacy for people present in the video stream and to protect other types of personal information that may be captured in the video stream.

[0003] As an example, object detection may be used to detect people. Masking of people may be done by extracting the image coordinates of the relevant portion of the image frame. Once the image coordinates are known, the relevant portion in the video stream may be masked, pixelated, blurred, or otherwise altered to obscure the identifier in the video stream.

[0004] One known way to achieve masking is to apply a Sobel filter, which results in an anonymized image in which only edges or contours are visible. The advantage of using this type of filter is that although the image is anonymized, the operator can still resolve the movements and actions of people in the captured scene, which is sufficient for many surveillance use cases.

[0005] However, compared to, for example, solid masking and pixelation, the bit rate of data generated from a video stream that has undergone a Sobel filter is higher, which may cause network or storage problems or general information overload for the operator.

[0006] As a result, there is room for improvement in dealing with the data generated in anonymized video streams. Summary of the Invention

[0007] In view of the above and other drawbacks of the prior art, it is an object of the present invention to provide an improved method for controlling the bit rate of a video stream captured with an image capture device and filtered using an edge filter, which mitigates at least some of the drawbacks of the prior art.

[0008] According to a first aspect of the present invention, there is therefore provided a method for controlling the bit rate of a video stream captured with an image capture device, the video stream being filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream.

[0009] The method includes determining a current bitrate value for the video stream, and in response to detecting that the current bitrate value exceeds a threshold, adjusting at least one parameter of an edge filter to further reduce the amount of edges in at least one area of the video stream, thereby controlling the bitrate.

[0010] The present invention is based on the recognition that adjusting the parameters of an edge filter changes the generated bit rate of the resulting filtered video stream. As a result, by monitoring the current bit rate and with knowledge of the allowable bit rate, the filter parameters of the edge filter can be adjusted to reduce the amount of edges, which can effectively control the bit rate. The bit rate is lowered as a result of reducing the amount of edges.

[0011] As a result, the inventors have recognized that using edge filters can mask areas of a video stream to adequately protect the privacy of people in the video stream while still allowing motion and action to be detected, and can also provide for actively, and at least near real-time, varying the parameters of the edge filters, thereby controlling bitrate.

[0012] A video stream is typically a set of consecutive image frames taken over time, which collectively form the video stream.

[0013] An edge filter operates using an algorithm that is adapted to find edges in an image frame. The basic operation of such an edge filter is that the algorithm finds edges or curves in the image frame where the image intensity has abrupt or sharp changes or discontinuities. Once the edges or curves are found, they are enhanced while suppressing the appearance of other parts. The edge filter is applied to each image frame in the video stream.

[0014] An edge filter is defined by parameters that may be part of a filter operator. The filter operator may be of different types depending on the particular edge filter being used. However, the filter operator often operates as a gradient operator that calculates or estimates the intensity gradient between adjacent pixels in an image frame. The adjacent pixels may be direct neighbors, but the filter operator often also includes second-neighbors and / or more distant neighbors. The calculated intensity gradient is compared to a detection threshold to determine whether an edge is considered to be detected.

[0015] As a result, an edge filter is a filter that enhances edge points and suppresses non-edge points. An edge point corresponds to one or more pixels in an image frame that are determined to belong to an edge, where the image intensity has an abrupt or sharp change or discontinuity. Similarly, a non-edge point corresponds to one or more pixels in an image frame that are determined not to belong to an edge. Thus, by applying an edge filter to a video stream, edge points in an image frame are enhanced while non-edge points are suppressed. For example, an edge filter, which is sometimes also called an edge detection filter, may be a Sobel filter, a Canny filter, a Difference of Gaussian (DoG) filter, a Laplacian filter, a Prewitt filter, a Roberts-Cross filter, or a Shar filter, just to name a few. To suit this use case, the output of the edge filter, i.e., the edge filtered version of the video stream, may be modified using one or more operations, such as an absolute value operation that does not distinguish between positive and negative gradients, a gain operator that adjusts intensity, an offset operator that makes edge points black and non-edge points white, and / or a noise reduction operator that removes unnecessary dark edge pixels or other structures in the image that are not part of a true edge.

[0016] One example edge filter is based on a 3x3 kernel or operator, where pixels in a 3x3 submatrix are considered in calculating the gradient for a given pixel, where the elements of the kernel set weights for different pixel values used to calculate the gradient. Using an operator of this size provides a computationally efficient edge filtering process while providing a filtered image of acceptable quality. Larger kernels, e.g., 4x4 or 5x5, are also applicable to the embodiments described herein.

[0017] The bit rate threshold may be set based on knowledge of the bit rate capacity or bandwidth of the network connected to the image capture device, or memory capacity, or some other predetermined parameter, and may include a margin to avoid risking exceeding some critical bit rate level.

[0018] The bit rate is the number of bits per unit time generated from an image frame of a video stream captured by an image capture device. The bits may be generated in an encoder that encodes the captured image frames according to an encoding protocol. The current bit rate value directly or indirectly indicates the bits generated per unit time.

[0019] Privacy masking is understood as a feature that protects a person's privacy by hiding or obscuring parts of an image frame with a masked area. Generally, privacy masks can be static or dynamic, with the most commonly described here being dynamic masks in the form of edge filters applied to the image frame, but also in the form of solid masks or blurs. Static masks may be applied globally in the same way across the entire image frame or at least a portion thereof, while dynamic masks may be applied once a face or person is detected in the video stream, for example.

[0020] The parameters of the edge filter may refer to the elements of the kernel or the detection threshold against which the output of the filter operator or gain operator of the edge filter is compared. Adjusting the parameters of the edge filter to reduce the current bitrate value may be understood as adjusting the protection need of the edge filter so that fewer edges are enhanced by the filter.

[0021] According to one embodiment, the current bitrate value may be the current bitrate or the rate of increase of the current bitrate. The rate of increase of the current bitrate is the rate or pace at which the bitrate is currently increasing. Monitoring and responding to the rate or pace at which the bitrate is currently increasing allows for a faster or more immediate response to high bitrates, since adjustments in the parameters of the edge filter can be made when a high rate of increase is detected before the absolute bitrate reaches a critically high bitrate. It may further be possible to combine the current bitrate with the rate of increase of the bitrate for a combined comparison with each of the thresholds. For example, if the rate of increase of the bitrate suddenly increases sharply, the filter parameters may subsequently be adjusted before the bitrate itself reaches the bitrate threshold.

[0022] According to one embodiment, the adjusting may further include adjusting at least one parameter of the edge filter until the current bitrate value is below the threshold, whereby the adjustment of the at least one filter parameter preferably continues until the bitrate is again at an acceptable level.

[0023] According to one embodiment, at least one parameter of the edge filter may be adjusted for a predetermined area of the captured scene. Determining the predetermined area may include detecting an area in the video stream where a person is present, for example, by using an object detection algorithm or motion detection. The predetermined area may also relate to an area in the video stream to which static masking should be applied. The predetermined area may be pre-set based on prior knowledge of the scene being monitored. Static masking relates to masking an area regardless of the presence of moving objects in that area. Thus, static masking is used to mask a predetermined area of the video stream.

[0024] According to one embodiment, the method may further include detecting an object in an area of the captured scene, detecting the size of the object in the captured scene, and adjusting at least one parameter of the edge filter differently in areas of the captured scene depending on the size of the detected object in each area. For example, privacy masking may be less important for people or objects farther from the image capture device because their privacy may be protected by being farther from the camera. People or objects farther from the image capture device appear smaller in the video stream. This means that the size of an object in the video stream provides an efficient way to determine whether or to what extent a particular object should be masked using an edge filter. This advantageously provides for tailoring the edge filter adjustment based on the privacy masking required in different areas of the video stream.

[0025] According to one embodiment, at least one parameter of the edge filter may be adjusted to further reduce the amount of edges in areas of a scene with objects having a detected size above or equal to a size threshold compared to areas with objects having a detected size below the size threshold. The size threshold may be set so that areas of the video stream where people are far enough away from the image capture device that they do not require excessive privacy masking are filtered while retaining more edges than areas where people are close to the image capture device. Preferably, filtering areas with objects close to the camera reduces the bit rate of the video stream more than filtering areas with objects further away, because objects in the video stream farther from the camera generally produce fewer edges than objects closer to the camera. This type of size thresholding provides a relatively simple yet efficient way of finding areas that require more edge filtering than other areas.

[0026] According to one embodiment, the adjusting may include adjusting parameters of an edge filter to globally statically mask an area of the captured scene, and thus at least one parameter of the edge filter may be adjusted for all areas defined as areas where static masking should be applied.

[0027] According to one embodiment, the method may further include detecting the number of people in the captured scene and adjusting at least one parameter of the edge filter in response to detecting that the number of people exceeds a threshold, i.e., the number of people threshold. If there are many people in the video stream, adjusted parameters of the edge filter may be required to reduce the bit rate. For example, if only people are masked using the edge filter and the number of people increases, the edge filter must subsequently be applied to more areas of the video stream, which increases the generated bit rate. Preferably, the parameters of the edge filter are therefore adjusted to accommodate this change in the number of people and the need for edge filtering in relation to the generated bit rate, thereby maintaining or reducing the generated bit rate or the rate of increase of the bit rate.

[0028] According to one embodiment, the method may further include applying image blur to selected areas of the captured scene to further control the bit rate.

[0029] According to one embodiment, the method may include applying solid color masking to selected areas of the captured scene to further control the bit rate.

[0030] For example, using blur or solid color masking provides additional tools to tailor privacy masking to the particular implementation at hand.

[0031] According to one embodiment, the method may further include detecting a moving object and adjusting at least one parameter of the edge filter according to the current size of the moving object in the captured scene. Moving objects, such as people, cause relatively high bit rates. Therefore, adjusting the edge filter is particularly suitable for areas in the video stream that contain moving objects.

[0032] In some possible implementations, the threshold may be a first threshold, whereby the method may further include, in response to detecting that the current bitrate value is less than a second threshold lower than the first threshold, adjusting at least one parameter of an edge filter to increase the amount of edges in at least one area of the video stream, thereby controlling the bitrate. Using the second, lower threshold serves to ensure that the edge filtering does not cause excessive loss of detail in the filtered video stream. As a result, the second threshold is set to be adjusted to enable the edge filtering to discern motion, events, or actions in the filtered video stream while still preserving privacy.

[0033] According to a second aspect of the present invention, there is provided a control unit for controlling a bit rate of a video stream captured with an image capture device and filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, the control unit being configured to perform the steps of the method according to the first aspect of the present invention.

[0034] Each of the further embodiments of this second aspect of the invention, and the advantages achieved through them, are largely similar to those described above for the first aspect of the invention.

[0035] According to a third aspect of the present invention, there is provided a system comprising an image capture device configured to capture a video stream and a control unit according to the second aspect.

[0036] The image capture device is preferably a video camera, for example a surveillance camera.

[0037] Each of the further embodiments of this third aspect of the invention, and the advantages achieved through them, are largely similar to those described above for the first and second aspects of the invention.

[0038] According to a fourth aspect of the present invention, there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of any one of the method embodiments described herein.

[0039] Each of the further embodiments of this fourth aspect of the invention, and the advantages achieved through them, are largely similar to those described above for the other aspects of the invention.

[0040] Further features and advantages of the present invention will become apparent upon review of the appended claims and the following description. Those skilled in the art will recognize that various features of the present invention may be combined to create other embodiments, each of which is described below, without departing from the scope of the present invention.

[0041] The various aspects of the present invention, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings set forth below. [Brief explanation of the drawings]

[0042] [Figure 1] 1 conceptually illustrates an example application of an embodiment of the present invention. [Figure 2]2 is a flowchart of method steps according to an embodiment of the present invention. [Figure 3A] 1 conceptually illustrates an image frame with an edge filter applied. [Figure 3B] 3B conceptually illustrates an image frame filtered using an edge filter having at least one adjusted parameter compared to the edge filter shown in FIG. 3A but used in FIG. 3A. [Figure 4] 2 is a flowchart of method steps according to an embodiment of the present invention. [Figure 5A-5B] A conceptual image frame is shown. [Figure 6] 2 is a flowchart of method steps according to an embodiment of the present invention. [Figures 7A-7B] 10A-10C are conceptual photographed image frames filtered with edge filters having different adjusted parameters. [Figure 8] 2 is a flowchart of method steps according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0043] The present invention will now be described in more detail with reference to the accompanying drawings, which illustrate presently preferred embodiments of the invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth below. Rather, these embodiments are provided for completeness and completeness, and to fully convey the scope of the invention to those skilled in the art. Like reference numerals refer to like elements throughout the drawings.

[0044] Turning now to these drawings, and in particular to Figure 1, there is seen a scene 1 that is monitored by an image capture device 200, for example a camera, or more particularly a surveillance camera. Scene 1 includes a set of objects 104a, 104b, which may be, for example, vehicles in a parking lot, and a number of people 106a-c.

[0045] Camera 200 may be mounted on a building, a pole, or any other suitable location depending on the particular application at hand. Furthermore, camera 200 may be a fixed camera, or a movable camera, such as a pan, tilt, and zoom camera, or may be a body-worn camera. Furthermore, camera 200 may be a visible light camera, an infrared (IR)-responsive camera, or a thermal (long-wave infrared (LWIR)) camera. Furthermore, image capture devices employing LIDAR and radar capabilities may also be considered.

[0046] Camera 200 continuously monitors scene 1 by capturing image frames that form a video stream of scene 1. Scene 1 within the camera's field of view is illustrated here as including objects 104a and 104b and people 106a-c. Camera 200 may transmit the video stream to client 116 or server 118 via wireless link 112, which is connected to communication network 114 having a certain transmission capacity, i.e., a certain bandwidth.

[0047] The camera 200 further includes an imaging module 202, an image processing pipeline 204, an encoder 206, a memory 208, and an input and output interface 210 configured as a communication interface between the camera 200 and the network 114 via the wireless link 112.

[0048] The imaging module 202 includes various components, such as a lens and an image sensor, where the lens is adapted to project an image, including a number of pixels, onto the image sensor.

[0049] Image processing pipeline 204 is configured to perform a range of different operations on image frames received from the image sensor. Such operations may include, for example, filtering, demosaicing, color correction, noise filtering to remove spatial and / or temporal noise, distortion correction to remove effects such as barrel distortion, global and / or local tone mapping to enable the capture of scenes containing a wide range of intensities, transformations such as rotation, flat-field correction to remove effects such as vignetting, application of overlays such as privacy masks or legends, etc. It should be noted, however, that some of these operations, e.g., transformation operations such as barrel distortion correction and rotation, may be performed by one or more modules, components, or circuits arranged outside of image processing pipeline 204, e.g., in one or more units between image processing pipeline 204 and encoder 206.

[0050] Following the image processing pipeline 204, the image frames are forwarded to an encoder 206, where they are encoded according to an encoding protocol and forwarded to a receiver, e.g., client 116 and / or server 118, via network 114 using input / output interface 210. It should be noted that the camera 200 shown in Figure 1 also includes many other components, such as a processor and memory, which are common in conventional camera systems and whose purpose and operation are well known to those skilled in the art. Such components have been omitted from the illustration and description of Figure 1 for reasons of clarity.

[0051] The camera 200 may also include a data storage 208 for storing data related to the capture of the video stream. This data storage may therefore store the captured video stream. This data storage may be a non-volatile memory, such as an SD card.

[0052] There are many conventional video encoding formats, some common video encoding formats that work with various embodiments of the present invention include JPEG, Motion JPEG (MJPEG), High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H Part 2, Advanced Video Coding (AVC), also known as H.264 and MPEG-4 Part 10, Versatile Video Coding (VVC), also known as H.266, MPEG-I Part 3, and Future Video Coding (FVC), VP9, VP10, and AOV Media Video 1 (AV1), just to name a few.

[0053] Typically, encoders use variable bitrate encoding formats, which means that data from an image frame that changes from frame to frame is processed by the encoder, and data that does not change, i.e., static areas in the image frame, does not need to be re-encoded. Thus, scenes with moving objects will incur a higher bitrate than scenes with no or few moving objects.

[0054] The image processing pipeline 204 and the encoder 206 may run on a control unit 212. The camera 200 and the control unit 212 may be part of a camera system 10, where the control unit 212 may be a separate, standalone control unit or part of the camera 200. It is conceivable that the control unit 212 resides remotely, for example on a server, and thus operates as a cloud-based service.

[0055] However, the control unit 212 and the imaging module 202 may be arranged as a single unit.

[0056] When monitoring scene 1, camera 200 continuously captures image frames using imaging module 202, processes the image frames in image processing pipeline 204, encodes the processed data in encoder 206, and, depending on the currently available bandwidth in communication network 114, stores the encoded image frames in data storage 208 and / or transmits them directly via communication network 114.

[0057] Depending on the nature of scene 1, edge filtering in the image processing pipeline 204 generates different amounts of data for the encoder 206 to encode. An excessively high bitrate may cause storage or network problems. For example, if the generated bitrate exceeds the bandwidth of the communication network 114, data transmission may be hindered. Furthermore, storing the encoded image frames may become difficult due to a lack of memory storage space in the data storage 208. Reducing the bitrate also reduces the load on the encoder 206, thereby avoiding network bandwidth problems on the communication network 114.

[0058] Edge filtering is a preferred type of privacy masking because it allows for privacy protection in the video stream while still allowing for recognition and detection of actions taking place in the video stream. However, using edge filtering in the image processing pipeline 204 may cause an increase in bitrate in the encoder 206, especially when object movement or motion is present in the video stream.

[0059] To alleviate this problem, the inventors propose to control the bit rate from the video stream captured with the image capture device 200 by adjusting the parameters of an edge filter, which is used for privacy masking of at least one area of the captured scene 1 in the video stream.

[0060] Turning now to FIG. 2, this shows a flow chart according to an embodiment of the present invention.

[0061] In step S102, the current bit rate value of the video stream is determined, which is determined by the encoder 206, which receives the edge-filtered video stream from the image processing pipeline 204.

[0062] The current bitrate value is compared to a threshold value, and in response to detecting in S103 that the current bitrate value exceeds the threshold value, at least one parameter of the edge filter is adjusted in step S104 to further reduce the amount of edges in at least one area of the video stream, thereby controlling the bitrate.

[0063] As a result, parameters of at least one edge filter are controlled to control the bit rate of the video stream data transmitted over the communications network 114. For example, if fewer edges are generated in the resulting encoded image, the bit rate will be reduced, i.e., fewer bits will be generated in the encoder 206 and less data will have to be transmitted over the communications network 114.

[0064] The comparison of the current bit rate value with the threshold value may be performed by processing circuitry within the control unit 212. Additionally, adjustment of the edge filter parameters may be performed in the image processing pipeline 204 where the edge filtering occurs.

[0065] 3A and 3B each show a conceptual example of an image frame 300 to which an edge filter with different levels of adjustment has been applied.

[0066] In Figure 3A, the image frame 300 includes the edges of two people 301 and 302, which are visible and relatively clear, but still have their privacy protected. In Figure 3A, an edge filter with a first level of protection or first adjustment setting has been applied, which allows the operator to determine the presence of the two people, but their privacy is protected.

[0067] Figure 3B shows the same image frame 300, but now with an edge filter applied that is less protective or with different adjustment settings than the edge filter applied to image frame 300 in Figure 3A, resulting in fewer edges being detected. Fewer edges results in a lower bit rate being introduced in encoder 206 because less data changes between successive image frames.

[0068] Different types of edge filters are applicable to the embodiment shown. One example edge filter relies on the Sobel filter operator, which is briefly described below.

[0069] The Sobel filter uses two 3x3 kernels, one horizontal TIFF0007719762000001.tif9170 change, the other is vertical The Sobel kernel may be given by: TIFF0007719762000003.tif21170

[0070] Both of these kernels are applied to a two-dimensional image frame by convolving the kernel with the two-dimensional image frame, which is represented by a matrix of pixel values, to obtain, for each pixel in the image frame, the gradient magnitude TIFF0007719762000004.tif7170 can be calculated as: TIFF0007719762000005.tif18170

[0071] The magnitude of this gradient TIFF0007719762000006.tif8170 is compared to a detection threshold to determine whether an edge is present at a given pixel. Adjusting the detection threshold adjusts the amount of edges present in the filtered video stream. For example, the detection threshold used to detect edges to produce the image frame shown in FIG. 3A was lower than the corresponding detection threshold used to produce the image frame shown in FIG. 3B, in which fewer edges were detected. Therefore, the higher detection threshold used in this edge filter resulted in fewer edges in FIG. 3B, resulting in a lower bit rate.

[0072] As described herein, other edge filter operators can also be applied using kernels of other sizes and other numerical elements contained in those kernels, the Sobel operator above being one example. Adjusting the elements of the kernel assigns different weights to the gradient centered on the central pixel currently under consideration. TIFF0007719762000007.tif8170 and Gives the pixel values used to calculate TIFF0007719762000008.tif9170.

[0073] Adjusting the parameters of the edge filter may amount to adjusting the threshold against which the gradient magnitude is compared, but it is also conceivable that adjusting the parameters of the edge filter will amount to adjusting the value of the edge filter kernel.

[0074] The control unit 212 may estimate the amount of adjustment needed for the edge filter based on the current bitrate value, or the adjustment may be fixed for each adjustment. If the edge filter adjustment is variable, the need for the current adjustment may be estimated based on the difference between the current bitrate value and a threshold. With variable bitrates, a bitrate control algorithm, such as maximum bitrate (MBR) or average bitrate (ABR), is typically used to still have some control over the bitrate over time. Both MBR and ABR employ variable bitrates, with MBR ensuring that the current bitrate is below an upper limit and thereby estimating the difference between the threshold and the current bitrate. ABR is configured to ensure that the average bitrate over a given time window is below a particular threshold. Both of these approaches allow for more sophisticated variation in the amount of adjustment based on the trend of bitrate change over time, associated with a given MBR or ABR.

[0075] Preferably, following adjusting the parameters of the edge filter in step S104, it is checked again in step S103 whether the current bitrate exceeds or is equal to the threshold. In this way, at least one parameter of the edge filter is adjusted until the current bitrate value is below the threshold. If a fixed adjustment is implemented, this fixed adjustment is applied until the current bitrate value is below the threshold. If a variable adjustment is implemented, the current room for adjustment is estimated before each adjustment.

[0076] The current bitrate value may be such current bitrate or a rate of increase of the current bitrate. Thus, the current bitrate value being monitored may be the current bitrate itself or the time gradient of the bitrate based on the change in bitrate from successive image frames.

[0077] At least one parameter of the edge filter may be adjusted for a given area of the captured scene. This may be implemented in a variety of ways, some of which are described below. For example, in areas containing moving objects, or in areas capturing distant people, or in areas of static privacy masking, the edge filter may be adjusted, while in other areas, the parameter of the edge filter may not be adjusted.

[0078] Now, look at the flow chart in FIG. 4 and the conceptual captured image frame 400 in FIGS. 5A and 5B.

[0079] Objects 106a-b in the area of the scene being photographed are detected in step S402. The objects may be detected and classified using an object detection algorithm, for example an object classifier, which may be operated in the image processing pipeline 204.

[0080] In step S404, the sizes of the objects 106a and b in the captured scene are detected. If the objects are far away, their size will be smaller than the size of objects closer to the camera 200. Therefore, detecting the size of the objects 106a and b provides an efficient way to at least indirectly determine whether they are far or close to the camera 200. Detecting the size may be done by estimation from the number of pixels associated with the detected object or by other detection algorithms.

[0081] Furthermore, details of an object 106b farther from the camera may not be readily discernible without an edge filter, so the edge filter over the corresponding area of the object 106b may be adjusted to reduce the number of edges more than in areas of the object closer to the camera 200. In this way, the bitrate is reduced without excessive loss of information because the actions of objects farther from the camera, i.e., the actions of a person, are not readily discernible regardless of edge filtering. However, preferably, an object farther from the camera 200, e.g., person 106b, may be far enough from the camera 200 that privacy masking for the person 106b is not necessary. A person 106a close to the camera 200 is close enough to require privacy masking of the corresponding region of the image frame in the video stream. Furthermore, due to the person 106a's proximity to the camera 200, more edges are detectable by the filter, so adjustment of the edge filter provides a greater reduction in bitrate value compared to a corresponding adjustment for an area of the image frame corresponding to the person 106b further away. Therefore, preferably, the edge filter over corresponding areas of object 106a that are close to the camera, i.e., of a size equal to or exceeding the size threshold, may be adjusted to reduce the number of edges more than that in areas of object 106b that are further away from the camera 200.

[0082] Thus, step S04 includes adjusting at least one parameter of the edge filter differently in areas of the captured scene depending on the size of the detected objects 106a and 106b in each of the areas. Figure 5A is a conceptual captured image frame 400, and Figure 5B conceptually shows image frame 400 after edge filtering adjustment. In the depicted example, the object 106b further from the camera is less filtered, i.e., some edges are still present or not edge filtered at all, while the area of object 106a closer to the camera is filtered to reduce the amount of edges more than the area of the further object 106b.

[0083] Size detection may be followed by a thresholding step, in which at least one parameter of an edge filter may be adjusted to further reduce the amount of edges for areas of the scene with objects having a detected size above or equal to the size threshold, compared to areas with objects having a detected size below the size threshold. The size threshold may be set depending on the use case for the camera 200.

[0084] As an alternative to detecting the size of an object, a depth model may be used to determine the distance from the detected object to the camera 200. This distance is compared to a distance threshold to determine whether to adjust at least one parameter of the edge filter.

[0085] The step S402 of detecting an object may include detecting whether the object is a person, and if the detected object is a person, an edge filter is applied to the area of the person, which may be adjusted based on the size of the detected person.

[0086] Furthermore, detecting an object in step S402 may further include detecting a moving object, and at least one parameter of the edge filter is adjusted differently according to a current size of the moving object in the captured scene.

[0087] In some embodiments, an area 500 may be predetermined for static masking, as conceptually shown in FIGS. 5A and 5B , where masking may be applied regardless of detected objects or motion. The static masking area 500 may relate to a fixed area where privacy masking is known in advance to be necessary, such as an area where monitoring is not permitted or where people are prohibited from entering and where other means of surveillance are not possible. The adjustment of the edge filter in such a static masking area 500 may be a global adjustment. In other words, the edge filter is adjusted globally for the entire area 500, as shown in FIG. 5B . However, the degree of adjustment of the edge filter parameters may depend on the distance from the camera of the area 500 corresponding to the static masking area.

[0088] As described herein, a large amount of motion in a captured scene generates a high bitrate. Therefore, since people tend to move in a scene, it is preferable to adjust the filtering according to the number of people in the scene. Therefore, in step S602 of the flowchart in Figure 6, the number of people in the captured scene is detected. Step S602 may be performed following or as part of step S402 in Figure 4.

[0089] 7A and 7B show subsequent image frames 701 and 702 in which an edge filter has been applied to moving people 106a and b in the first image frame 701. However, as the number of people increases and exceeds a threshold, e.g., a people threshold of three in this example, at least one parameter of the edge filter is adjusted in step S104 to further reduce the number of edges of detected people 106a-d, as shown in Figure 7B. Thus, despite the increased amount of motion in the video stream caused by the increased number of people, the bitrate can be maintained below the threshold.

[0090] Other types of masking may be applied as a complement to edge filters. For example, image blurring may be applied to selected areas of the captured scene to further control the bitrate. In another example, a solid color mask may be applied to selected portions of the captured scene to further control the bitrate. Applying a solid mask requires less computational power and memory than applying an edge filter, which may be advantageously used for privacy masking of areas where action detectability is undesirable or not possible, such as areas far from the camera 200.

[0091] If the bitrate is at a relatively low value, this may then allow for an increase in the bitrate, allowing an operator at the client 116 or server 118 to adjust the parameters of the edge filter to increase the amount of edges. This may enhance the detail in the video stream to better resolve any action in the video stream. To this end, as shown in the flowchart of FIG. 8, the method may further include, in response to detecting in step S803 that the current bitrate value is less than a second threshold value that is lower than the first threshold value, adjusting in step S804 at least one parameter of the edge filter to increase the amount of edges in at least one area of the video stream, thereby controlling the bitrate. Once this adjustment is made in step S804, the method again checks whether the bitrate exceeds the first threshold value, e.g., the threshold described in connection with FIG. 2 in step S103. The second threshold value is selected so that some level of detail is still present in the filtered video stream, allowing action and events in the video stream to still be resolved. In other words, the second threshold is selected such that actions, events, or motion are resolvable or discernible in the filtered video stream while still preserving privacy for objects and people in the video stream. The second threshold may be set for the current use case.

[0092] There is further provided a control unit 212 for controlling the bit rate of a video stream captured using an image capture device and filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, the control unit being configured to perform each of the method steps described herein.

[0093] There is provided a computer program comprising instructions which, when executed by a computer, cause the computer to perform any one of the methods described herein.

[0094] The control unit may include a microprocessor, a microcontrol unit, a programmable digital signal processor, or another programmable device. The control unit may also, or instead, include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. The control unit may include a programmable device, such as a microprocessor, a microcontrol unit, or a programmable digital signal processor as described above, which may further include computer executable code for controlling the operation of the programmable device.

[0095] The control functions of the present disclosure may be implemented using an existing computer processor, or by a specialized computer processor incorporated for this or another purpose for an appropriate system, or by a hardwired system. Embodiments within the scope of the present disclosure include program products including machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media may be any available medium that can be accessed by a general-purpose or special-purpose computer or other machine with a processor. By way of example, such machine-readable media may include RAM, ROM, EPROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to hold or store desired program code in the form of machine-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (whether hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Accordingly, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

[0096] While the figures may show a sequence, the order of steps may differ from that depicted. Also, two or more steps may occur simultaneously or partially simultaneously. Such variations will depend on the software and hardware systems selected and the designer's choice. All such variations are within the scope of this disclosure. Similarly, software implementations may be achieved using standard programming techniques involving rule-based logic and other logic to accomplish the various connection, processing, comparison, and decision steps. Additionally, while the present invention has been described with reference to specific exemplary embodiments thereof, many different modifications, variations, and the like will be apparent to those skilled in the art.

[0097] Furthermore, variations to the disclosed embodiments can be understood and effected by the addressee, skilled in the art, in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. Moreover, in the claims, the word "comprising" does not exclude other elements or steps. The indefinite article "a" or "an" does not exclude a plurality.

Claims

1. 1. A method for controlling a bit rate of a video stream captured with an image capture device, the video stream being filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, the method comprising: determining a current bitrate value of the video stream; In response to detecting that the current bitrate value exceeds or is equal to a threshold value, detecting objects in the area of the photographed scene; Detecting the size of the object in the photographed scene; adjusting at least one parameter of the edge filter to further reduce the amount of edges in the at least one area of the video stream, thereby controlling the bitrate; wherein the adjusting includes adjusting the at least one parameter of the edge filter differently in the areas of the photographed scene depending on the size of the detected object in each of the areas.

2. The method of claim 1 , wherein the current bit rate value is a current bit rate or an increase rate of the current bit rate.

3. The method of claim 1 , comprising adjusting the at least one parameter of the edge filter until the current bit rate value is less than the threshold value.

4. The method of claim 1 , wherein the at least one parameter of the edge filter is adjusted for a predetermined area of the captured scene in the video stream.

5. 2. The method of claim 1, wherein the at least one parameter of the edge filter is adjusted to further reduce the amount of edges for areas of the photographed scene having detected objects with sizes above or equal to a size threshold compared to areas having detected objects with sizes below the size threshold.

6. The method of claim 1 , wherein the adjusting comprises adjusting the parameters of the edge filter to globally statically mask an area of the photographed scene.

7. The method of claim 1 , further comprising applying image blur to selected areas of the captured scene to further control the bit rate.

8. 10. The method of claim 1, further comprising applying solid color masking to selected areas of the photographed scene to further control the bit rate.

9. 9. A control unit for controlling the bit rate of a video stream captured with an image capture device and filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, the control unit being configured to perform the method of any one of claims 1 to 8.

10. The control unit of claim 9 , further comprising an image capture device configured to capture a video stream.

11. A non-transitory computer-readable storage medium storing a computer program, the program including instructions that, when executed by a computer, cause the computer to perform a method according to any one of claims 1 to 8.

12. A method for controlling the bit rate of a video stream captured with an image capture device, the video stream being filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, comprising: determining a current bitrate value of the video stream; Detecting the number of people in the captured scene; in response to detecting that the number of people exceeds a first threshold, adjusting at least one parameter of the edge filter to further reduce an amount of edges of the detected people in the video stream, thereby controlling the bitrate; in response to detecting that the current bitrate value exceeds or is equal to a second threshold, adjusting at least one parameter of the edge filter to further reduce an amount of edges in the at least one area of the video stream, thereby controlling the bitrate; A method comprising:

13. A method for controlling the bit rate of a video stream captured with an image capture device, the video stream being filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, comprising: determining a current bitrate value of the video stream; Detecting a moving object; in response to detecting that the current bitrate value exceeds or is equal to a threshold, adjusting at least one parameter of the edge filter according to a current size of the moving object in the captured scene to further reduce the amount of edges in the at least one area of the video stream, thereby controlling the bitrate; A method comprising:

14. A method for controlling the bit rate of a video stream captured with an image capture device, the video stream being filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, comprising: determining a current bitrate value of the video stream; in response to detecting that the current bitrate value exceeds or is equal to a first threshold, adjusting at least one parameter of the edge filter to further reduce an amount of edges in the at least one area of the video stream, thereby controlling the bitrate; in response to detecting that the current bitrate value is less than a second threshold lower than the first threshold, adjusting at least one parameter of the edge filter to increase an amount of edges in the at least one area of the video stream, thereby controlling the bitrate; A method comprising:

15. A control unit for controlling the bit rate of a video stream captured using an image capture device and filtered using an edge filter for privacy masking of at least one area of a captured scene in the video stream, the control unit being configured to perform a method described in any one of claims 12 to 14.

16. A non-transitory computer-readable storage medium storing a computer program, the program including instructions that, when executed by a computer, cause the computer to perform a method according to any one of claims 12 to 14.

Images