Ptz masking control

Abstract

To provide a method of generating an output video stream with privacy masking when receiving multiple video streams from multiple cameras capturing the same scene, a corresponding device, a camera monitoring system, and a non-transitory computer-readable storage medium including a computer program.SOLUTION: A method includes receiving a first input video stream from at least one first camera capturing a scene, receiving a second input video stream from a second camera that only partially captures the scene, and generating an output video stream 220 from the first input video stream. In response to determining that a certain area 212 of the output video stream depicts a portion 210 of the scene depicted in the second input video stream, the method discontinues masking 230b, 230c of the certain area of the output video stream or at least reduces the level of the masking.SELECTED DRAWING: Figure 2C

Description

[Technical Field]

[0001] This disclosure relates to (privacy) masking in video streams generated in camera surveillance systems, and in particular to such masking when receiving multiple video streams from multiple cameras capturing the same scene. [Background technology]

[0002] In modern camera surveillance systems, multiple cameras can be used to capture the same scene. For example, one or more cameras can be used to generate an overview of the scene, while one or more other cameras can be used to generate a more detailed view of the scene. The camera used to generate the scene overview can be, for example, a camera with a larger field of view (FOV) so that it can capture a larger portion of the scene. Another example includes combining video streams from multiple cameras, each with a smaller FOV and each capturing a different portion of the scene, to form a combined video stream that still captures a larger portion of the scene. To generate a more detailed view of the scene, a so-called pan-tilt-zoom (PTZ) camera can preferably be used, which can, for example, zoom in on an object or area of ​​interest in the scene and can provide more detail about such object or area than would be available from the scene overview alone. A user of the camera surveillance system can then, for example, use an overview of the scene to identify the presence of an object or area of ​​interest, and then use the PTZ camera to investigate the object or area in more detail (e.g., by zooming in closer to the object of interest and / or by using the pan and / or tilt functions of the PTZ camera to follow the object of interest as it moves across the scene).

[0003] In such camera surveillance systems, it is not uncommon for different users to have different levels of authority, and a higher level of authority may be required to see a more detailed view of a scene than is required to see only an overview of the scene. This is because the scene overview provides less detail about individual objects (such as people) than a PTZ camera. In particular, to further reduce the level of authority required to see the scene overview, (privacy) masking can be applied to the scene overview, such that, for example, the identities of people or license plates are hidden and cannot be recovered by a user with lower level of authority viewing the scene overview. On the other hand, in the case of a PTZ camera, the full potential of the PTZ camera can be utilized if an object of interest is found, and therefore, a higher level of authority is typically required to see a more detailed view of a scene generated by a PTZ camera, and such masking is typically not performed.

[0004] When a PTZ camera is used, for example, to follow ongoing events (e.g., to track the movement of an object of interest), it may be desirable to also use a scene overview to improve situational awareness. For example, if a PTZ camera is used to zoom in on an object of interest, it may be difficult to accurately recognize where the object is currently located within the scene if only a more detailed view of the scene is used. By also using a scene overview, such difficulty in locating an object within the scene may be alleviated. However, if privacy masking is performed in the scene overview, it may be difficult to utilize the full potential of the scene overview for situational awareness. Summary of the Invention

[0005] To remedy the above situation where (privacy) masking of objects and / or regions in the scene overview is used to reduce the required level of permissions needed to view the scene overview, but where it is also desirable to use the scene overview to improve situational awareness, the present disclosure provides an improved method for generating an output video stream, as well as corresponding devices, camera surveillance systems, computer programs and computer program products as defined in the accompanying independent claims. Various embodiments of the improved methods, devices, camera surveillance systems, computer programs and computer program products are defined in the accompanying dependent claims.

[0006] According to a first aspect, a method for generating an output video stream is provided. The method includes receiving a first input video stream from at least one first camera capturing a scene. The method further includes receiving a second input video stream from a second camera only partially capturing the scene. The method also includes generating the output video stream from the first input video stream, the output video stream including ceasing masking or at least reducing a level of masking for a particular region of the output video stream in response to determining that the particular region of the output video stream depicts a portion of the scene depicted in the second input video stream. In some embodiments, it is contemplated that masking can be ceased (or reduced) for regions of the output video stream corresponding to portions of the scene depicted in the second input video stream, while normal (i.e., not ceased or reduced) masking can occur for one or more other regions of the output video stream that do not correspond to portions of the scene depicted in the second input video stream. Note that "cease masking" or "reduce the level of masking" implies that such regions will actually be masked in the output video stream if no contemplated method is utilized that takes into account the content of the second input video stream.

[0007] The envisioned method improves upon currently available techniques and solutions in that it takes into account what is currently seen in the second input video stream from the second camera and then prevents (or reduces the level of) masking of corresponding regions in the scene overview, thereby improving situational awareness, for example, by making objects of interest currently being tracked in a more detailed view of the scene visible in the scene overview, and in particular by making such "unmasking" in the scene overview dynamic and adapting to changes in the FOV of the second camera.

[0008] In some embodiments of the method, determining that a particular region of the output video stream depicts a portion of a scene depicted in the second input video stream may include comparing the field of view (FOV) of at least one first camera with the field of view of a second camera. As used herein, a camera's FOV may include, for example, the camera's position relative to the scene or at least relative to another camera, the camera's zoom level (optical or digital), or the camera's tilt / pan / rotation. Thus, by comparing the FOVs of two cameras, it is possible to determine whether the two cameras are capturing the same portion of the scene, and in particular, which region of the image captured by one camera corresponds to the portion of the scene captured by the other camera. In other embodiments, it is contemplated that other techniques may be used to achieve the same effect, and that other techniques may be based, for example, on image analysis and feature identification instead of camera FOV parameters.

[0009] In some embodiments of the method, the method may include generating an output video stream from multiple first input video streams from multiple first cameras, each capturing a portion of a scene. A particular region of the output video stream may capture all of the scene captured by at least one of the multiple first cameras (e.g., as captured in at least one of the multiple first input video streams). For example, it is contemplated that two or more cameras may be positioned such that they are pointed in different directions and thus capture different portions of the scene (possibly with some overlap, but not necessarily). The input video streams from these cameras may then be combined into a single composite video stream on which the video output stream is based. The video streams from each camera may be stitched together to create the illusion of receiving only a single input video stream, for example, from a very large FOV camera. In other embodiments, the input video streams from each first camera may instead be shown adjacent to each other in the output video stream, for example, in a horizontal array, vertical array, and / or grid.

[0010] In some embodiments of the method, the masking may include pixelating certain regions of the output video stream. Reducing the level of masking may include reducing the block size used to pixelate certain regions of the output video stream. This may be advantageous in that, for example, the resolution of an object of interest may be the same in both the output video stream and the second output video stream that represents the second input video stream.

[0011] In some embodiments of the method, the method may further include determining, based on the first input video stream, that an extended region of the output video stream that includes the specific region is a region to be masked. The method may include masking at least a portion of the extended region outside the specific region in the output video stream. In other words, it is contemplated that one may still retain a portion of a larger mask that would otherwise be desired, and "unmask" only a specific region of such larger mask that corresponds to a portion of the scene that is also captured by the second camera.

[0012] In some embodiments of the method, the extended region of the output video stream may be a predetermined region of the first input video stream, or, stated another way, the extended region of the output video stream corresponds to a predetermined portion of a scene. For example, such a predetermined region / portion of a scene may be manually set when configuring a camera surveillance system, e.g., to statically mask one or more features or areas of a scene where people are typically present or known in advance to be inhabited. Such features and / or areas may be, for example, windows of residential buildings, windows of coffee shops or restaurants, daycare centers, playgrounds, outside service areas, parking lots, bus or train stops, etc.

[0013] In some embodiments of the method, the portion of the scene captured in the second input video stream may be defined as a portion of the scene that includes at least one object of interest. In other words, otherwise desired masking of the first input video stream and the output video stream may be more dynamic and not static, e.g., following people or other objects as they move through the scene, with masking occurring only after it has been further determined that people and / or objects should otherwise be masked (if their masking should not cease according to the intended method). For example, masking of an object may be desired only if it has first been determined (e.g., by using object detection, etc.) that the object belongs to a class (e.g., person, license plate, etc.) that should be masked.

[0014] In some embodiments of the method, the portion of the scene captured in the second input video stream may be defined as all of the scene captured within a predetermined area of ​​the second input video stream, such as the entire area of ​​the second input video stream. In other embodiments, the predetermined area of ​​the second input video stream may correspond to an area smaller than the entire area of ​​the second input video stream. For example, it may be envisioned to detect in which area of ​​the second input video stream, for example, an object of interest is located, and unmask (or rather, cease masking) only the portion of the first input video stream in the output video stream that corresponds to this area, thus "unmasking" an area of ​​the scene in the output video stream that is smaller than the area captured by the second camera.

[0015] In some embodiments of the method, the method may be performed in a one-camera configuration. The camera configuration may include at least one first camera and a second camera. In other embodiments, it is envisioned that at least one first camera and one second camera are provided as separate cameras that do not form part of the same camera configuration, and the method is then performed by any of the cameras, or, for example, by all of the cameras jointly (assuming the cameras have some method of communication with each other).

[0016] According to a second aspect of the present disclosure, there is provided a device for generating an output video stream, the device including a processor, the device further including a memory storing instructions that, when executed by the processor, cause the device to receive a first input video stream from at least one first camera capturing a scene, receive a second input video stream from a second camera only partially capturing the scene, and generate an output video stream from the first input video stream, the output video stream including ceasing to mask, or at least reducing a level of masking, a particular region of the output video stream in response to determining that the particular region of the output video stream captures a portion of the scene captured in the second input video stream.

[0017] Thus, a device according to the second aspect is configured to perform the corresponding steps of the method of the first aspect.

[0018] In some device embodiments, the device is further configured to perform any of the method embodiments described herein (i.e., such that the instructions, when executed by the processor, cause it to do so).

[0019] In some embodiments of the device, the device is one of at least one first camera and a second camera. In other embodiments of the device, the device is a camera configuration that includes all of at least one first camera and a second camera.

[0020] According to a third aspect of the present disclosure, there is provided a camera surveillance system, the camera surveillance system including at least a first camera, a second camera, and a device as contemplated herein according to the second aspect (or any embodiment thereof).

[0021] According to a fourth aspect of the present disclosure, there is provided a computer program for generating an output video stream. The computer program product, when executed by a processor of a device (e.g., a device according to the second aspect or any embodiment thereof), is configured to cause the device to receive a first input video stream from at least one first camera capturing a scene, receive a second input video stream from a second camera only partially capturing the scene, and generate an output video stream from the first input video stream, wherein, in response to determining that a particular region of the output video stream depicts a portion of the scene depicted in the second input video stream, the output video stream includes ceasing masking, or at least reducing a level of masking, of the particular region of the output video stream.

[0022] Thus, a computer program according to the fourth aspect is configured to cause a device to perform a method according to the first aspect.

[0023] In some embodiments of the computer program, the computer program is further configured to cause the device to perform any embodiment of the method of the first aspect described herein.

[0024] According to a fifth aspect of the present disclosure, there is provided a computer program product. The computer program product includes a computer-readable storage medium storing a computer program according to the fourth aspect (or any embodiment thereof described herein). The computer-readable storage medium may, for example, be non-transitory and may be provided, for example, as a hard disk drive (HDD), a solid-state drive (SDD), a USB flash drive, an SD card, a CD / DVD, and / or any other storage medium capable of non-transitory storage of data.

[0025] Other objects and advantages of the present disclosure will become apparent from the following detailed description, drawings, and claims. Within the scope of the present disclosure, it is contemplated that, for example, all features and advantages described with reference to the method of the first aspect may relate to, apply to, and be used in combination with any features and advantages described with reference to the device of the second aspect, the surveillance camera system of the third aspect, the computer program of the fourth aspect, and / or the computer program product of the fifth aspect, and vice versa.

[0026] Exemplary embodiments will now be described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0027] [Figure 1] 1 illustrates a schematic diagram of a typical camera surveillance system. [Figure 2A] 1 illustrates a schematic diagram of an embodiment for generating an output video stream according to the present disclosure. [Figure 2B] 1 shows a schematic flow chart of an embodiment of a method according to the present disclosure. [Figure 2C] 10 illustrates schematically an alternative embodiment of a method according to the present disclosure. [Figure 3] 1A-1C illustrate various embodiments of a device for generating an output video stream according to the present disclosure; DETAILED DESCRIPTION OF THE INVENTION

[0028] In the drawings, like reference numerals are used for like elements unless otherwise specified. Unless explicitly stated to the contrary, the drawings show only the elements necessary to illustrate the exemplary embodiments, while other elements may be omitted or merely suggested for clarity. As shown in the figures, the sizes (absolute or relative) of elements and regions may be increased or decreased relative to their true values ​​for illustrative purposes, and are therefore provided to illustrate the general structure of the embodiments.

[0029] 1 schematically illustrates a typical camera surveillance system 100, including a camera configuration 110, a device 150, and a user terminal 160. The camera configuration 110 includes four first cameras 120 (only three of which, 120a, 120b, and 120c, are visible in FIG. 1 ) and a second camera 130. The first cameras 120 are overview cameras mounted in a circular pattern, each facing a different direction so that together they capture, for example, a 360-degree horizontal view of a scene. The second camera 130 is a pan-tilt-zoom (PTZ) camera mounted below the first cameras 120, in the center of the circular pattern (e.g., when viewed from above or below).

[0030] A first camera 120 generates a first input video stream 140 (i.e., if there are four first cameras 120, then four input video streams 140a-d), and a second camera generates a second input video stream 142. The input video streams 140, 142 are provided to a device 150. The device 150 is configured to receive the input video streams 140, 142 and generate corresponding output video streams 152, 154.

[0031] Output video streams 150, 152 are provided by device 150 to user terminal 160, with first output video stream 152 displayed on first monitor 161 of terminal 160 and second output video stream 154 displayed on second monitor 162 of terminal 160. In the particular example of system 100 shown in FIG. 1 , four first input video streams 140a-d (one from each camera 120) are provided to device 150, which combines first input video streams 140a-d into a single combined first output video stream 152. For example, device 150 may arrange input video streams 140a-d in a 2×2 grid, with each quadrant of the 2×2 grid displaying one first input video stream from a corresponding first camera. Of course, other alternative configurations for combining such multiple first input video streams into a single combined first output video stream are also contemplated. For example, various input primary video streams may be stitched together to create a panoramic output primary video stream.

[0032] It may be assumed herein that device 150 is a standalone device separate from camera arrangement 110 and user terminal 160. In other embodiments, device 150 may instead be an integrated part of either camera arrangement 110 or user terminal 160.

[0033] A user of terminal 160 may use first output video stream 152 displayed on first monitor 161 for situational awareness, while second output video stream 154 displayed on second monitor 162 can be used to provide more detail about a particular portion of a scene. For example, a user may detect an object or area of ​​interest by looking at first monitor 161 and then adjust the field of view (FOV) of second camera 130 so that second camera 130 captures the object or area of ​​interest in more detail. For example, the FOV of second camera 130 can be adjusted, e.g., by zooming, panning, rotating, and / or tilting second camera 130 accordingly, so that second output video stream 154 displayed on second monitor 162 provides a zoomed-in view of the particular object or area of ​​interest. To gain situational awareness, first output video stream 152 displayed on first monitor 161 can be used, e.g., to still track where a zoomed-in object is currently located within a scene.

[0034] The adjustment of the FOV of the second camera 130 may also be automated such that, if an object or region of interest is detected in one or more first input video streams 140 and / or corresponding first output video stream 152 (e.g., by using an object detection and / or object tracking algorithm), the FOV of the second camera 130 is adjusted such that the second input video stream 142 (and thereby the corresponding second output video stream 154) provides an enlarged view of the detected object or region of interest when displayed on the second monitor 162. Instead of using physical / optical zoom-in (i.e., by adjusting the lens of the second camera 130), it may also be envisioned to use, for example, a digital zoom, such that the second input video stream 142 is not altered.

[0035] For privacy reasons, it may be required (e.g., by local law) to mask one or more “sensitive” portions of the scene so that, for example, the identity or other personal characteristics of objects or vehicles present in the sensitive portions of the scene cannot be derived from viewing the first monitor 161. Such sensitive portions may correspond, for example, to locations where people are likely to be present in the scene, such as shopping windows, windows of residential buildings, bus or train stops, restaurants, or coffee shops. Accordingly, for this purpose, the camera system 100 and device 150 are configured to apply one or more privacy masks when generating the first output video stream 152. In the particular example shown in FIG. 1 , three such privacy masks 170a, 170b, and 170c are applied. The first privacy mask 170a is in front of the portion of the scene including a store, the second privacy mask 170b is in front of the portion of the scene including a bus stop, and the third privacy mask 170c is in front of a residential flat located on the lowest / ground floor of the building. 1, these privacy masks are shown as diagonal lines, but of course they may instead be completely opaque so that no detail about, for example, a person behind the mask can be obtained by viewing the first monitor 161. In other embodiments, the privacy masks 170a-c may be created, for example, by pixelating the relevant area, or even created so that the object to be masked is transparent (this is possible, for example, if an image of the scene without the object before it is available). Other ways of masking objects are also envisioned.

[0036] 1 , a particular object of interest 180 is present in the first input video stream 140a and constitutes a person currently attempting to break into a store. Because the person 180 appears to have smashed through a shopping window in the store, the user of the terminal 160 is interested in tracking the progress of the break-in so that the person's 180 location can be followed in real time. For example, by adjusting the FOV of the second camera 130 accordingly, a close-up view of the person 180 is provided in the second output video stream 154 displayed on the second monitor 162. No privacy mask is applied in the second output video stream 154.

[0037] As previously described herein, system 100 is often configured so that the level of authorization required to view first output video stream 152 is lower than the level of authorization required to view second output video stream 154. This is because the scene overview provided by first output video stream 152 contains less detail about individual objects in the scene; i.e., because privacy masks 170a-c are applied before the user can view first output video stream 152 on first monitor 161. However, in situations such as that shown in FIG. 1 , it may be desirable for the user of terminal 160 to use first output video stream 152 for situational awareness so that the user can more easily determine where person 180 is currently located in the scene. For example, if person 180 begins to move, it may be difficult to determine where person 180 is currently located in the scene simply by looking at second terminal 162. Knowing person 180's current location may be important, for example, when contacting law enforcement. However, due to the privacy masks 170a-c, such situational awareness is hindered as the masks 170a-c hide the person 180 in the first output video stream 152, thus making it more difficult to utilize the first output video stream 152 displayed on the first monitor 161 for such situational awareness.

[0038] How the methods envisaged herein can help to improve the above situation will be explained in more detail below, also with reference to Figures 2A-2C.

[0039] FIG. 2A illustrates a possible embodiment of a method 200 for generating an output video stream, and FIG. 2B illustrates a flowchart of various steps involved in such method 200. In a first step S201, a first input video stream 140 is received from at least one first camera (e.g., one or more of cameras 120a-120c), where the first input video stream 140 captures / views a scene. In a second step (S202, which may be performed before or simultaneously with step S201), a second input video stream 142 is received from a second camera (e.g., camera 130). The second input video stream 142 captures / views only a portion of the scene. For example, the FOV of the second camera 130 is smaller than the FOV (or combined FOV) of the first camera (or multiple first cameras) 120. In other words, the second input video stream 142 provides a magnified view of an object or area of ​​the scene. As previously described herein, such a result can be obtained, for example, by mechanically / optically zooming the second camera 130 (i.e., by moving the lens), or, for example, by digitally zooming. In the latter case, the second input video stream 142 produced by the second camera 130 is not necessarily modified to produce the second output video stream 154, although for purposes of describing the envisioned improved method, it can be assumed in any case that the second input video stream 142 thus only partially captures the scene.

[0040] The envisaged method 200 then proceeds to generate an output video stream 220 from the first input video stream 140 in a third step S203, as described below. To generate the output video stream 220, it is first determined whether a particular region 212 of the resulting output video stream 220 depicts a portion 210 of a scene that is depicted in the second input video stream 142. If so, the generation of the output video stream 220 ceases, or at least reduces, the level of masking of the particular region 212 of the output video stream.

[0041] 2A , the second input video stream 142 includes a zoomed-in view of the person 180 and depicts a portion of the scene that is also found in the first input video stream 140, as indicated by the dashed rectangle 210. To find the location and size of the rectangle 210, it is contemplated that, for example, knowledge of the current FOV of the second camera 130 and knowledge of the current FOV of the one or more first cameras 120 can be used and compared. In the contemplated method 200, it can then be determined that a particular region 212 of the corresponding output video stream 220 depicts the same portion of the scene as the second input video stream 142, i.e., the portion indicated by the dashed rectangle 210. As used herein, having knowledge of a camera's FOV includes, for example, knowing the camera's position relative to the scene (and / or relative to one or more other cameras), the camera's zoom level (optical / mechanical and / or digital), and the camera's orientation (e.g., the camera's pan, tilt, and / or roll), etc. Thus, by comparing the FOV of the two cameras, it is possible to calculate, for example, whether the first camera sees the same part of the scene as seen by the second camera, and in particular which area of ​​the video stream produced by the first camera corresponds to this same part of the scene.

[0042] As already described with reference to FIG. 1 , when applying one or more privacy masks 230 a- 230 c to mask, for example, shops, bus stops, and residential flats, the envisioned method 200 does not apply masking to specific regions 212 of the output video stream 220. As a result, situational awareness of the scene obtainable from the output video stream 220 is therefore enhanced because there is no mask covering the person 180. If the person 180, for example, starts moving, the person 180 will not be masked in the output video stream 142 as long as the person 180 remains within the FOV of the second camera 130 generating the second input video stream 220. For example, it may be envisioned that the second camera 130 tracks the person 180 (by manual control by a user or, for example, using object detection and / or object tracking), and that the specific regions 212 of the output video stream 220 are therefore updated as the FOV of the second camera 130 changes, such that the person 180 remains unmasked in the output video stream 220. When using the envisioned method to improve the general camera surveillance system 100 shown in FIG. 1, the device 150 may be configured, for example, to perform the improved method 200 and, for example, to replace the first output video stream 152 with the output video stream 220 so that the output video stream 220 is displayed on the first monitor 161 instead of the first output video stream 152.

[0043] 2A , the particular region 212 of the output video stream 220 where masking is discontinued corresponds to the entire portion of the scene captured / imaged in the second input video stream 142. In other words, the portion of the scene captured in the second input video stream 142 is defined as all of the scene captured in a predetermined region of the second input video stream 142, such as the entire region of the second input video stream 142. Of course, in other embodiments, the particular region 212 of the output video stream 220 where masking is discontinued can correspond only to the portion of the scene captured / imaged in the second input video stream 142. For example, it may be envisioned to discontinue masking only 90%, 80%, 75%, etc., of the portion of the scene captured / imaged in the second input video stream 142. In such a situation, the predetermined region of the second input video stream 142 may be smaller than the entire region of the second input video stream 142. In any embodiment, what is important is that the masking ceases in a sufficient area of ​​the output video stream 220 to allow objects and / or areas of the scene of interest to be visible in the output video stream 220.

[0044] 2C schematically illustrates another embodiment of the envisioned method, showing only the resulting output video stream 220. In the output video stream 220, masking is instead discontinued in all of the scene captured by one of the first input cameras 120 (i.e., in regions of the output video stream 220 that may correspond to a larger portion of the scene than that captured by the second camera 130). In this case, the portion of the scene captured in the second input video stream 142 corresponds only to the region marked by the dashed rectangle 210, but the particular region 212 of the output video stream where masking is discontinued is a region that captures all of the scene captured by one of the cameras 120. In other words, in this embodiment, the first input video stream 140 is a composite video stream from multiple first cameras 120 (e.g., cameras 120a-c), each capturing a portion of the scene, and the particular region 212 of the output video stream 220 captures all of the scene captured by at least one of the multiple first cameras 120. In other situations, for example, where the dashed rectangle 210 spans input video streams from multiple first cameras 120, it may be assumed, for example, to discontinue masking in all of the portions of the scene captured by the corresponding first cameras 120.

[0045] 2C , as already described with reference to, e.g., FIG. 2A , the present disclosure also contemplates that the particular region 212 of the output video stream 220 is part of a larger, extended region of the output video stream 220 to which it would otherwise be determined that masking should be applied. In other words, after ceasing masking within the particular region 212, there may be other regions of the output video stream 220 where masking is still performed, e.g., such that at least a portion of the extended region outside the particular region 212 of the output video stream 220 is still masked. Thus, an advantage of the proposed contemplated method 200 is that it allows for improved situational awareness without, e.g., unmasking regions of the scene that are not related to the currently tracked object of interest; i.e., the privacy of the object may still be respected as long as its position in the scene does not coincide with the tracked object.

[0046] As used herein, the extension region of the output video stream 220 may be, for example, a predetermined region of the first input video stream 140. For example, the extension region may be statically defined, e.g., by a user manually drawing / marking one or more polygons to which masking should be applied, e.g., using a video management system (VMS). This may be useful, for example, when the first camera 120 is stationary and always captures the same part of the scene. In other embodiments, it is envisioned that the extension region (excluding certain areas) to which masking should be applied may instead be dynamic and change over time, e.g., based on object detection and / or object tracking algorithms that analyze the various input video streams to detect / track objects that are classified as sensitive and to which masking should normally be applied. Thus, an envisioned method allows for overriding such masking decisions when the second camera 130 is currently following the object of interest, so that the object of interest remains unmasked in the scene overview provided by the output image stream 220 (and, for example, when displayed on the first monitor 162 of the user terminal 160).

[0047] In another embodiment of the method 200, it is contemplated that instead of completely eliminating masking in a particular region 212, the level of masking is instead modified (i.e., reduced). In such an embodiment, masking is typically performed by increasing the level of pixelation of the object. For example, by dividing an input image of the input image stream into blocks of MxM pixels (M is an integer), each pixel in such a block may be replaced with the average pixel value in the block. Since a pixel block of size M=1 corresponds to no masking / pixelation at all, and if M is equal to the total width and / or height of the image, the pixel block renders all of the image as having the same color (corresponding to the average color of all the original pixels), increasing the size of the block (i.e., increasing M) can reduce the level of detail in the image. In the method contemplated herein, it may be assumed that masking is performed by performing such MxM block averaging in at least the region of the output video stream 220 where the sensitive region or object is located. For example, masking may be performed with a pixelation block size of size 8x8. Thus, in a particular region 212 of the output video stream 220, a level of masking can be obtained by reducing the block size used for pixelation, for example to 4x4 or 2x2 (or even 1x1, which is equivalent to removing masking completely). This is particularly useful when the resolution of one or more of the first cameras 120 is greater than the resolution of the second camera 130, for example, because the level of masking in the output video stream 220 is adapted to match the resolution level of the second camera 130 (on the first monitor 161). By doing so, the user of the terminal 160 will see the object of interest 180 at the same resolution on both the first monitor 161 and the second monitor 162.As an example, using the camera surveillance system 100 shown in FIG. 1 , it can be assumed that each of the four first cameras 120 has 4K resolution (e.g., 3840×2160) and the second camera 130 has 1080p resolution (e.g., 1920×1080). The first cameras 120 can be positioned, for example, to each cover a 90-degree horizontal FOV (thus, together creating a 360-degree view of the scene, without taking into account any overlap between the first input video streams 140). In other embodiments, there may be only a single 4K resolution first camera 120 positioned to cover a 90-degree horizontal FOV. The second camera 130 may be configured to cover a 90-degree horizontal FOV when fully zoomed out. As a result, the pixel density of the second camera 130 corresponds to a 2×2 pixelation block size in the output video stream 220. If the second camera 130 is zoomed in, the corresponding pixelation block size in the output video stream 220 will instead be 1×1, e.g., no pixelation / masking will be performed. Thus, in a situation where the second camera 130 is fully zoomed out and masking is normally performed by averaging over pixel block sizes, e.g., 4×4, 8×8, or 16×16, the level of masking in a particular region 212 of the output video stream 220 can be adjusted by reducing the block size to match the block size of the second camera 130, e.g., 2×2. Thus, the pixel density of the person 180 will be the same when viewed on both the first monitor 161 and the second monitor 162 of the terminal 160.

[0048] It is contemplated herein that method 200 may be performed, for example, by (or within) a camera surveillance system (e.g., camera surveillance system 100 shown in FIG. 1 , but with device 150 reconfigured / substituted to perform method 200) and / or by a device such as those described in more detail below, also with reference to FIG. 3. Such a camera surveillance system is provided by the present disclosure.

[0049] 3 schematically illustrates a device 300 for generating an output video stream according to one or more embodiments of the present disclosure. The device 300 includes at least a processor (or “processing circuitry”) 310 and a memory 312. As used herein, a “processor” or “processing circuitry” may be any combination of one or more suitable central processing units (CPUs), multiprocessors, microcontrollers (μCs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), graphics processing units (GPUs), etc., capable of executing software instructions stored in the memory 312. The memory 312 may be external to the processor 310 or may be internal to the processor 310. As used herein, “memory” may be any combination of random access memory (RAM) and read-only memory (ROM), or any other type of memory capable of storing instructions. The memory 312 includes (i.e., stores) instructions that, when executed by the processor 310, cause the device 300 to perform the methods described herein (i.e., method 200 or any embodiment thereof). Device 300 may further include one or more additional items 314 that may be necessary in some circumstances to perform the method. In some exemplary embodiments, device 300 may be, for example, a camera configuration as shown in FIG. 1 , where one or more additional items 314 may then include, for example, one or more first cameras 120 and second cameras 130. One or more additional items 314 may also include, for example, various other electronic components necessary to generate the output video stream. Performing the method within a camera assembly may be useful, for example, in that processing is moved to the “edge,” i.e., closer to where the actual scene is captured, compared to performing the method elsewhere (such as on a more centralized processing server). Device 300 may be connected to a network, for example, so that the generated output video stream can be transmitted to a user (terminal).To this end, device 300 may include a network interface 316, which may be, for example, a wireless network interface (e.g., one supporting Wi-Fi, e.g., as defined by IEEE 802.11 or any successor standard) or a wired network interface (e.g., one supporting Ethernet, e.g., as defined by IEEE 802.3 or any successor standard). Network interface 316 may also support any other wireless standard capable of transferring encoded video, such as Bluetooth. The various components 310, 312, 314, and 316 (if present) may be connected via one or more communication buses 318 so that these components can communicate with each other and exchange data as necessary.

[0050] A camera configuration such as that shown in FIG. 1 is advantageous because all cameras are provided as a single unit and because it may be easier to compare the FOVs of various cameras since the relative positions of the cameras are already known from the factory. It is also contemplated that the camera configuration may include the circuitry needed to perform method 200, such as device 300 as described above, and may output, for example, output video stream 220 and possibly second output video stream 152, such that the camera configuration may be directly connected to a user terminal without the need for any intermediate device / processing system. In other embodiments, the device for performing method 200 contemplated herein may be a separate entity that may be provided between the camera configuration and a user terminal. Such camera configurations including the functionality to perform contemplated method 200 are also provided by the present disclosure.

[0051] For example, it is also contemplated herein that at least one first camera 120 may be provided as its own entity / unit, and that second camera 130 may also be provided as a different, separate entity / unit. For example, a fisheye camera may be used to provide an overview of a scene, such as mounted on the ceiling of a building, while a PTZ camera may be mounted elsewhere (e.g., on a wall) and used to provide a more detailed view of the scene. In such a configuration, a device contemplated herein may be provided, for example, as part of one or more first cameras used to generate an overview of the scene, or as part of a second camera used to generate a more detailed view of the scene. In other contemplated embodiments, the device may instead be provided as a separate entity or as part of a user terminal, and configured to receive respective input video streams from the various cameras and generate one or more output video streams.

[0052] The present disclosure also contemplates providing a computer program and corresponding computer program product as previously described herein.

[0053] To summarize the various embodiments presented herein, the present disclosure provides an improved way of generating an output video stream including a scene overview in a system in which an additional second camera is present to generate a more detailed view of a portion of the scene. By ceasing masking, or at least reducing the level of masking, in the output video stream including the scene overview based on which portion of the scene is currently captured by the second camera, the use of the output video stream providing the scene overview for situational awareness is improved, and the output video stream can thus be used to, for example, better determine the location or other whereabouts of objects or areas of interest captured by the second camera as the objects move within the scene.

[0054] Although the features and elements may be described above in particular combinations, each feature or element may be used alone without other features and elements, or in various combinations with or without other features and elements. Further, variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0055] In the claims, the words "comprising" and "including" do not exclude other elements, and the indefinite articles "a" or "an" do not exclude a plurality. The mere fact that certain features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage. [Explanation of symbols]

[0056] 100 camera surveillance system 110 Camera Configuration 120(ac) First Camera 130 Second Camera 140(ad) First input video stream 142 Second Input Video Stream 150 devices 152 First Output Video Stream 154 Secondary Output Video Stream 160 User Terminals 161 First Monitor 162 Second Monitor 170a-c (Privacy) mask applied to first output video stream 180 Target Objects 200 Improved Methods S201-203 Improved Method Steps 210 the portion of the scene captured by the second camera 212 Specific Regions of the Output Video Stream 220 output video stream 230a-c (Privacy) mask applied to the enhanced output video stream 300 Improved Devices 310 processor 312 memory 314 Optionally, one or more additional items 316 Optional Network Interface 318 one or more communication buses

Claims

1. A method (200) for generating an output video stream, comprising: Receiving (S201) a first input video stream (140) from at least one first camera (120a-120c) capturing a scene; receiving (S202) a second input video stream (142) from a second camera (130) that only partially captures the scene; and generating (S203) an output video stream (220) from the first input video stream, the output video stream including, in response to determining that a particular region (212) of the output video stream depicts a portion of the scene depicted in the second input video stream, ceasing masking (230a-230c) of the particular region of the output video stream or at least reducing a level of masking (230a-230c) and performing normal masking in one or more other regions of the output video stream that do not correspond to the portion of the scene depicted in the second input video stream; A method comprising:

2. 2. The method of claim 1 , wherein determining that the particular region of the output video stream captures the portion of the scene captured in the second input video stream comprises comparing a field of view of the at least one first camera with a field of view of the at least one second camera.

3. 2. The method of claim 1, comprising generating the output video stream from a plurality of first input video streams (140a-140d) from a plurality of first cameras (120a-120c) each capturing a portion of the scene, wherein the particular region of the output video stream depicts all of the scene captured by at least one of the plurality of first cameras.

4. 2. The method of claim 1 , wherein masking comprises pixelating the particular region of the output video stream, and wherein reducing a level of the masking comprises reducing a block size used for the pixelating of the particular region of the output video stream.

5. 2. The method of claim 1, comprising: determining, based on the first input video stream, that an extended region of the output video stream that includes the particular region is a region to be masked; and masking at least a portion of the extended region outside the particular region in the output video stream.

6. The method of claim 5 , wherein the extended region of the output video stream is a predetermined region of the first input video stream.

7. 2. The method of claim 1, wherein the portion of the scene captured in the second input video stream is defined as all of the scene captured within a predetermined area of ​​the second input video stream, such as the entire area of ​​the second input video stream.

8. The method of claim 1 , wherein the method is performed with a camera configuration including the at least one first camera and the second camera.

9. 1. A device for generating an output video stream, comprising: A processor; Memory for storing instructions the instructions, when executed by the processor, cause the device to: receiving a first input video stream from at least one first camera capturing a scene; receiving a second input video stream from a second camera that only partially captures the scene; and generating an output video stream from the first input video stream, the output video stream including, in response to determining that a particular region of the output video stream depicts a portion of the scene depicted in the second input video stream, ceasing masking, or at least reducing a level of masking, of the particular region of the output video stream and performing normal masking in one or more other regions of the output video stream that do not correspond to the portion of the scene depicted in the second input video stream. A device that causes

10. The device of claim 9, wherein the instructions, when executed by the processor, further cause the device to perform the method of any one of claims 2 to 8.

11. 10. The device of claim 9, wherein the device is one of the at least one first camera and the second camera, or includes one of the at least one first camera and the second camera.

12. A camera surveillance system comprising at least one first camera, a second camera, and the device of claim 9.

13. 1. A computer program for generating an output video stream, the computer program, when executed by a processor of a device, causing the device to: receiving a first input video stream from at least one first camera capturing a scene; receiving a second input video stream from a second camera that only partially captures the scene; and generating an output video stream from the first input video stream, the output video stream including, in response to determining that a particular region of the output video stream depicts a portion of the scene depicted in the second input video stream, ceasing masking, or at least reducing a level of masking, of the particular region of the output video stream and performing normal masking in one or more other regions of the output video stream that do not correspond to the portion of the scene depicted in the second input video stream. A computer program configured to cause a computer to 14. The computer program of claim 13, further configured to cause the device to perform a method according to any one of claims 2 to 8.

15. A non-transitory computer-readable storage medium storing the computer program of claim 13.

Images