Method for handling video streams and monitoring camera

Abstract

To provide a method for generating a discontinuous panoramic image from first and second video streams, and a monitoring camera.SOLUTION: A method obtains first and second video streams having an overlapping region, performs hard handover for a moving object when the moving object is detected in the overlapping region, and applies a blending algorithm to image data other than the moving object in the video stream within the overlapping region. At this time, the blending algorithm is disabled for the image data of the moving object.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure generally relates to surveillance cameras and methods performed by surveillance cameras. More particularly, the present disclosure relates to handling a first video stream and a second video stream. For example, the present disclosure relates to handling a first video stream and a second video stream.

Background Art

[0002] Panoramic images, panoramic video streams, and panoramic cameras are being used in ever more applications, and the field is thus currently evolving. Video surveillance is an application example of panoramic images, panoramic video streams, and panoramic cameras. A panoramic camera can capture an image or video stream of a scene and include one or more adjacent image sensors configured to have overlapping fields of view. Then, images from different image sensors are stitched together to form one image, i.e., a panoramic image. Such a panoramic image may be referred to as a stitched panoramic image. There are seams, for example stitch lines, between the overlapping images.

[0003] The process of image stitching consists of several steps, and image blending is one of them. There are several types of image blending, such as alpha blending, multiband blending, and the like. Image blending, simply blending, is performed in the overlap region or areas of adjacent images. In a camera with two image sensors, the image sensors on both sides of the seam contribute in different degrees. For example, when passing from left to right in the overlap region, the image sensor on the left side of the seam may be blended so as to decrease linearly from 100% to 0%, and the image sensor on the right side of the seam may be blended so as to increase linearly from 0% to 100%.

[0004] In cameras with multiple image sensors, alignment misalignment is generally optimized at a considerable distance from the camera, corresponding to the background in the scene, making the misalignment more noticeable. Moving objects in the scene generally move closer to the camera, which can cause discontinuities as the moving object crosses the overlapping area. This can be distracting for the viewer and make it difficult to correctly perceive and identify moving objects.

[0005] Therefore, at the very least, this problem needs to be mitigated or resolved. [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Therefore, the purpose of this disclosure is to avoid at least one of the above-mentioned drawbacks and to improve the processing of the first and second video streams. [Means for solving the problem]

[0007] According to a first aspect, the objective is achieved by a method performed by a surveillance camera for handling a first video stream and a second video stream. The surveillance camera comprises a first image sensor and a second image sensor. The surveillance camera acquires a first video stream from the first image sensor and a second video stream from the second image sensor. The field of view of the first video stream partially overlaps with the field of view of the second video stream within an overlapping region. The surveillance camera identifies a moving object within the overlapping region and divides the moving object into a plurality of sub-objects. The surveillance camera performs each hard handover of the sub-objects from the first video stream to the second video stream when each sub-object is entirely within the overlapping region and when a point of each sub-object passes through the boundary within the overlapping region.

[0008] According to a second aspect, the objective is achieved by a surveillance camera configured to handle a first video stream and a second video stream. The surveillance camera comprises a first image sensor and a second image sensor. The surveillance camera is configured to obtain a first video stream from the first image sensor and a second video stream from the second image sensor. The field of view of the first video stream partially overlaps with the field of view of the second video stream within an overlapping region. The surveillance camera is configured to identify a moving object within the overlapping region and divide the moving object into a plurality of sub-objects. The surveillance camera is configured to perform each hard handover of the sub-objects from the first video stream to the second video stream when each sub-object is entirely within the overlapping region and a point of each sub-object passes through the boundary within the overlapping region. [Effects of the Invention]

[0009] By dividing a moving object into multiple sub-objects and performing a hard handover for each sub-object, it is possible to identify and recognize the sub-objects that are considered to be related parts of the moving object. Therefore, the handling of the first and second video streams is improved.

[0010] This disclosure in this specification offers many advantages, and a non-exclusive list of these advantages is as follows:

[0011] The advantages of the disclosures herein are that it is easier to correctly identify subobjects and that the reliability of subobject recognition is increased.

[0012] Another benefit of this disclosure is the improved stitching of multiple video streams. The resulting video stream, i.e., the stitched video stream, contains sub-objects in a format that is useful in scenarios where only the sub-objects of interest are relevant, and the rest of the moving object is not of interest. This improves the recognition of sub-objects in the video stream.

[0013] The disclosures herein are not limited to the features and advantages described above. Those skilled in the art will recognize further features and advantages by reading the detailed description below. [Brief explanation of the drawing]

[0014] Herein, this disclosure is described in further detail only by reference to the attached drawings and in the following detailed description. [Figure 1] This is a schematic block diagram showing a surveillance camera according to the present invention. [Figure 2a] This is a schematic diagram showing two video streams according to the present invention. [Figure 2b] This is a schematic diagram showing two video streams according to the present invention. [Figure 2c] This is a schematic diagram illustrating the video stream before handover according to the present invention. [Figure 2d] This is a schematic diagram showing an example of a first video stream 201a and a second video stream 201b captured by the first image sensor 103a and the second image sensor 103b, respectively, according to the present invention. [Figure 3] This is a schematic diagram of the video stream after handover according to the present invention, and a flowchart showing the method. [Figure 4] This is a schematic block diagram showing a surveillance camera according to the present invention.

[0015] The drawings are not necessarily to scale, and in order to make them clear, the dimensions of certain features may be exaggerated. Instead, emphasis is placed on illustrating the principles of the present disclosure.

Mode for Carrying Out the Invention

[0016] FIG. 1 shows an example of a surveillance camera 100. The surveillance camera 100 may be a camera adapted to monitor its surroundings, for example, a scene.

[0017] The surveillance camera 100 may be any suitable camera adapted to capture and record its surroundings, for example, surrounding images, video streams, etc. The surveillance camera 100 may be referred to as a video camera, digital camera, digital surveillance camera, digital video camera, surveillance camera, security camera, image capture device, video stream capture device, etc. When referring to the surveillance camera 100, the term camera may be used for simplicity.

[0018] The surveillance camera 100 includes a plurality of image sensors 103, for example, a first image sensor 103a and a second image sensor 103b. Note that two image sensors 103 are used as an example in FIG. 1, and the surveillance camera 100 may include any n image sensors. Here, n is a positive integer. Here, when referring to either the first image capturing device 103a or the second image capturing device 103b, the reference number 103 is used. The first image sensor 103a and the second image sensor 103b may be of the same or different image sensor types.

[0019] Each image sensor 103 is adapted to sense its surroundings, that is, to sense light and convert it into a video stream. That is, it is to capture the video stream of the scene. The video stream may be described as a plurality of images, a recording of a set of images, etc.

[0020] The image sensors 103 configured within the surveillance camera 100 may be configured to communicate with each other using any suitable wired or wireless communication technology.

[0021] The first image sensor 103a and the second image sensor 103b are positioned within the surveillance camera 100 so as to have overlapping fields of view. Also, the first image sensor 103a and the second image sensor 103b may be arranged horizontally with respect to the ground, or they may be arranged vertically, or they may be arranged in any other suitable manner. The first image sensor 103a and the second image sensor 103b may be adjacent to each other, that is, they may be adjacent.

[0022] Figures 2a and 2b are schematic diagrams showing an example of video streams captured by the first image sensor 103a and the second image sensor 103b. Figure 2a shows a bus and Figure 2b shows the license plate on the bus. The scene where the video stream is captured by the first image sensor 103a and the second image sensor 103b may be a street where a bus and a car are driving, a street where people are walking, etc. The first image sensor 103a captures the first video.

[0023] The video stream 201a and the second image sensor 103b capture the second video stream 201b. In the example shown in the figure, in Figures 2a and 2b, the first image sensor 103a is located on the left side of the second image sensor 103b, and the two image sensors 103a, 103b are arranged horizontally with respect to the ground. As a result, the first video stream 201a is located on the left side of the second video stream 201b. Even though it is a figure. Figures 2a and 2b show an example having two video streams 201, and the present disclosure is equally applicable to any m video streams, where m is an integer greater than 1.

[0024] There is an overlapping region 203 between the first video stream 201a and the second video stream 201b. The overlapping region 203 may be referred to as an overlay region, joint region, or intersection region. The overlapping region 203 is the region where the fields of view of the first image sensor 103a and the second image sensor 103b overlap. In other words, the first video stream 201 and the second video stream 201beach have image data that represents or shows the same, for example, the same license plate, the same part of a scene, etc., but perhaps from slightly different angles due to parallax, for example. The first video stream 201a and the second video stream 201beach have a region outside or in addition to the overlapping region 203. That is, the first image sensor 103a and the second image sensor 103b have a non-overlapping field of view in addition to the overlapping region 203.

[0025] The moving object 205 is identified within the overlapping area 203 and can be exemplified by a bus traveling on a road or a person walking on a sidewalk. In Figure 2a, the moving object 205 is exemplified by a bus moving in the direction indicated by the arrow. The moving object 205 may extend outside the overlapping area 203 in at least one direction, i.e., it may not fit within the overlapping area. The moving object 205 may extend outside the overlapping area 203 in two directions, both to the right and to the left of the overlapping area 203. For example, a bus may be too wide to fit within the overlapping area 203.

[0026] First, the bus is captured by the first image sensor 103a, which generates the first video stream 201a. After the bus moves a certain distance from left to right, it enters the field of view of the second image sensor 103b, which generates the second video stream 201b. If the bus is in the overlap region 203, it is captured by both the first image sensor 103a and the second image sensor 103b. If the license plate 205 is completely outside the overlap region 203, it is captured by only one of the image sensors 103, i.e., either the first image sensor 103a or the second image sensor 203b.

[0027] If the moving object 205 does not fit within the overlapping region 203, for example, if a bus is identified, the moving object 205 is divided into multiple sub-objects 210. Let us use the example shown in the figure. 2a, the license plate may be one sub-object 210, and the driver's face may be another sub-object 210. Sub-objects 210 can be called blobs, partial objects, etc. Sub-objects 210 can have any appropriate shape, such as a circle, square, triangle, irregular shape, or the same shape as the license plate or the driver's face.

[0028] Depending on how large the overlapping region 203 is for an interesting sub-object 210, it may be necessary to further subdivide the sub-object 210. For example, if the sub-object 210 is exemplified by a license plate, and the entire license plate does not fit within the overlapping region 203, as illustrated in the figure, then 2b, it may be necessary to treat each letter or number on the plate, or groups of letters or numbers, as individual sub-objects 210. This means that both the moving object 205 and the sub-objects 210 may extend outside the overlapping region. The interesting sub-object 210 may be subject to further processing or analysis for surveillance purposes, such as license plate recognition or facial recognition.

[0029] As the bus moves from left to right as shown by the arrows in the figure, 2b, the license plate, being attached to the bus, also moves in the same direction. First, the license plate is captured by the first image sensor 103a, which generates the first video stream 201a. After the bus has moved a certain distance from left to right, it enters the field of view of the second image sensor 103b, which generates the second video stream 201b. If the license plate is in the overlap region 203, it is captured by both the first image sensor 103a and the second image sensor 103b. If the license plate is completely outside the overlap region 203, it is captured by only one of the image sensors 103, i.e., either the first image sensor 103a or the second image sensor 103b.

[0030] When a first video stream 201a and a second video stream 201b are obtained, the two video streams are stitched together to form a resulting video stream, namely a panoramic video stream. The first video stream 201a and the second video stream 201b can be the source video streams, and the panoramic video stream can be the target video stream. The stitching of video streams can be described as the joining, merging, or other similar operations of video streams.

[0031] As the bus and, consequently, the license plate move from left to right, as indicated by the arrows in the figure, the sub-objects 2a and 2b also move in the same direction. As the two video streams 201 are stitched together, and each sub-object 210 moves toward and across the boundary 208 within the overlapping region 203, each sub-object 210 is initially represented by image data from only one of the image sensors 101, e.g., the first image sensor 103a. Once the points of each sub-object 210 reach the boundary 208, they begin to be represented by image data from only the other image sensor 103, e.g., the second image sensor 103b. This can be described as performing a hard handover of each sub-object 210 from the first video stream 201a to the second video stream 201b. The term "hard" can refer to the fact that each sub-object 210 is initially represented by image data from the first video stream 201a in a single instance, and then at the next instance by image data from the second video stream 201b, in other words, there is no gradual or soft handover from the first video stream 201a to the second video stream 201b, where image data from both the first video stream 201a and the second video stream 201b are used to represent the sub-object 210. A hard handover occurs between two frames, for example, two adjacent frames. The sub-object 210 is represented by image data from either the first video stream 201a or the second video stream 201b.

[0032] If sub-object 210 moves in the direction indicated by the arrow in the figure, for example, 2a and 2b mean that sub-object 210 will first be on one side of boundary 208, for example the left side, and then gradually pass through boundary 208 until it is completely on the other side of boundary 208, for example the right side. When sub-object 210 is on one side of boundary 208, for example the left side, and before it passes through boundary 208, it can be represented by image data from the first video stream 201a. When a point within sub-object 210 passes through boundary 208, sub-object 210 may be handed over so that it is completely represented by image data from the second video stream 201b. This point can be located at any suitable location within sub-object 210, for example, at the center of sub-object 210, at a certain distance from one end or both ends of sub-object 210, or at one end of sub-object 210, etc. Therefore, the sub-object 210 may be handed over so that it completely passes through the boundary 208 and is then represented by the image data contained in the second video stream 201b, or so that only a portion of the sub-object 210 passes through the boundary 208 and is then represented by the image data contained in the second video stream 201b. The boundary 208 may extend through part or all of the overlap region 203, or it may be a point within the overlap region 203. The boundary 208 may be located in the region of the overlap region 203 corresponding to the location of the sub-object 210, or it may be located in both the region corresponding to the location of the sub-object 210 and the region where the sub-object 210 is not located. The boundary 208 may extend through the overlap region 203 in any direction, for example, horizontal, vertical, diagonal, etc., and the direction may correspond to the direction in which the first image sensor 103a and the second image sensor 103b are positioned relative to each other.

[0033] Further details regarding the handover of image data from the first video stream 201a to the second video stream 201b are described below with reference to Figures 2c and 2d. Figure 2c is a schematic diagram showing an example of the first video stream 201a and the second video stream 201b captured by the first image sensor 103a and the second image sensor 103b, respectively. Figure 2c shows video stream 201 obtained in double instance t and t+1. Figure 2d shows video stream 201 obtained in double instance t+2. Figure 2c shows an example where the sub-object 210 is a license plate on a bus moving from left to right, as seen by the arrows in the figure. Note that the horizontal positions of the first camera sensor 103a and the second camera sensor 103b are used only as an example to illustrate the principle, and the first camera sensor 103a and the second camera sensor 103b may be positioned in any other suitable manner, such as vertically or diagonally.

[0034] In Figure 2c, the overlapping region 203 is indicated by a dotted line. As sub-object 210 moves from left to right in the illustrated direction, referring to the arrow in Figure 2c, sub-object 210 is initially represented by the image data of the first video stream 201a. After a certain distance, sub-object 210 begins to pass through the boundary 208, which is indicated by a thick vertical line in the figure. 2c. Top drawing of the figure. 2c shows the position of sub-object 210 at time t when it begins to pass through the boundary 208. Bottom drawing of the figure. 2c shows the position of sub-object 210 at time instance t+1. Thus, top and bottom drawings in the figure. 2c illustrates sub-object 21O at different time instances. Drawing 2d in the figure shows the position of sub-object 210 at time instance t+2, which is a later time instance than the bottom drawing in the figure (Figure 2c).

[0035] The seam 213 is shown as a thick dotted line in the figure. In 2c, in the portion of the overlapping region 203 where the sub-object 210 is not located, the seam 213 passes through the overlapping region 203 along a line that essentially corresponds to the boundary 208. In the portion of the overlapping region 203 that constitutes the sub-object 210, the seam 213 then moves and adaptively follows the sub-object 210. The seam 213 can follow a first end, exemplified by the right end in the figure. Figure 2c shows the case when the sub-object 210 has moved. Comparing the upper and lower parts of Figure 2c, it can be seen that the seam 213 adapts to the movement of the sub-object 210. As the sub-object 210 moves in the same direction, the seam 213 moves closer to the end of the overlapping region 203.

[0036] Both the first video stream 201a and the second video stream 201b have the same seam 213 in the top view representing time t. Therefore, the seam 213 is the same when the first video stream 201a and the second video stream 201b represent the same frame in time, as shown in the bottom diagram. In 2c, representing time instance t+1, the first video stream 201a and the second video stream 201b also have the same seam 213, but it is a different seam from the one in the top diagram. The seam 213 differs in that it adapts to the movement of the sub-object 210.

[0037] Prior to the handover, at times t and t+1, sub-object 210 is represented by image data from the first video stream 210a, which lies on one side of the seam 213, as illustrated on the left in the figure. Image data from the opposite side of the seam 213, illustrated on the right in Figure 2c, is not used in the resulting panoramic image because it is redundant.

[0038] The sub-object 210 is represented by image data from the first video stream 201a on one side of the seam 213, for example, the left side, until a point within the sub-object 210 passes through the boundary 208. As described above, the point can be located at any preferred position within the sub-object 210, for example, at the center of the sub-object 210, at a certain distance from one end or both ends of the sub-object 210, or at one end of the sub-object 210.

[0039] When a point of sub-object 210 passes through boundary 208, the sub-object is handed over from the first video stream 201a to the second video stream 201b. This point is, for example, in the position illustrated in the bottom view of the figure. After the handover in Figure 2c, at time t+2, as illustrated in the figure, the seam 213 is adapted or moved to adaptively follow the second end, for example, the left end, as sub-object 210 moves in the direction of the arrow. The rest of the seam 213 still substantially corresponds to boundary 208, as before the handover. After the handover, for example, at time t+2 as shown in Figure 2d, sub-object 210 is represented by image data from the second video stream 201b on the second side of the seam 213, for example, the right side.

[0040] In the first video stream 201a, there may be a distance between the seam 213 that adaptively surrounds the first end of the sub-object 210 and the corresponding end of the overlapping region 203. Similarly, for the second video stream 201b, there may be a distance between the seam 213 that adaptively surrounds the second end of the sub-object 210 and the corresponding end of the overlapping region 203.

[0041] In the process of stitching video streams 201a and 201b together, the image data, i.e., pixels, from the first video stream 201a and the second video stream 201b may or may not be blended within the overlapping region 203. Image data from the first video stream 201a and the second video stream 201b that do not represent sub-object 210 may be blended. This may correspond to or surround the outer boundary 208 of the sub-object 210 that can be blended, as illustrated by the diagonally shaded region in Figures 2c and 2d. When image data from the first video stream 201a is blended with image data from the second video stream 201, the image data is joined together, for example, using a percentage of the image data from both video streams 201 and 201b. In the resulting panoramic video stream, the areas where blending has been performed may appear as sharp images, or as some sharp areas and some blurred areas. The blurred areas may be due to parallax or lining errors. The overlapping region 203, i.e., the sub-object 210, which has not undergone blending, does not need to be blurred, but it may appear to flicker from one video stream to another. In other words, the sub-object 210 may appear clear in the resulting panoramic video stream, while the areas above and below the sub-object 210 around the boundary 208 may be blurred.

[0042] Blending can be performed using a blending algorithm. The use of a blending algorithm on image data excluding sub-objects 210 may be the default setting, i.e., it may be turned on when the stitching procedure is performed, or the use of a blending algorithm may be determined when the method is performed, or it may be determined for each sub-object 210. Any suitable type of blending algorithm can be used, for example, alpha blending, multiband blending, etc.

[0043] In another example, the bus driver's face may be a moving object 205 of interest. Different moving objects 205 may be classified as belonging to a particular object class. Moving objects 205 may be prioritized or unprioritized objects. For example, if a bus has several moving objects 205 such as a license plate and the driver's face, the license plate may be a prioritized object, while the driver's face may be an unprioritized object. Thus, the license plate may be a moving object that is subject to sub-object splitting and hard handover. The driver's face may not be subject to sub-object splitting and hard handover, or it may be subject to sub-object splitting and hard handover later, that is, after the license plate has been subject to sub-object splitting and hard handover. Prioritization may be predetermined by the user.

[0044] Here, we will explain the method described above from the perspective of the surveillance camera 100. Figure 3 is a flowchart illustrating this method.

[0045] A surveillance camera 100 for handling a first video stream 201a and a second video stream 201b. The surveillance camera 100 includes a first image sensor 103a and a second image sensor 103b. The steps shown in the dotted box in Figure 3 are optional steps. This method includes at least one of the following steps performed by the surveillance camera 100, which may be performed in a more appropriate order than those described later:

[0046] Step 301 The surveillance camera 100 acquires a first video stream 201a from a first image sensor 103a and a second video stream 201b from a second image sensor 103b. The field of view of the first video stream 201a partially overlaps with the field of view of the second video stream 201b within the overlap region 203. The first video stream 201a and the second video stream 201b can be obtained directly from their respective image sensors 103 or via some other entity, such as local memory or cloud memory configured within the surveillance camera 100.

[0047] Step 302 The surveillance camera 100 identifies a moving object 205 within the overlapping area 203. The moving object may be, for example, a bus driving down the street or a person walking. The moving object 205 may extend outside the overlapping area 203 in at least one direction.

[0048] Step 303 The surveillance camera 100 can obtain information indicating prioritized and unprioritized sub-objects.

[0049] The moving object 205 may belong to an object class. An object class can be associated with prioritized and unprioritized subobjects. The multiple subobjects that are split from the moving object 205 in step 308 and which are subjected to a hard handover in step 309 may be prioritized subobjects. In the example where the bus is the moving object 205, the license plate may be a prioritized subobject, and the driver's face may be an unprioritized subobject. In another example, the license plate may be a prioritized subobject, and the headlight next to the license plate may be an unprioritized subobject.

[0050] Information indicating prioritized and unprioritized sub-objects can be obtained from locally configured memory within the surveillance camera 100 and from the cloud. Memory, central memory storage device, or other suitable memory. Information may be predetermined by the operator of the surveillance camera 100, provided to the surveillance camera 100 during its operation, or obtained by the surveillance camera 100 upon request or in any other suitable manner.

[0051] Step 304 The surveillance camera 100 divides the moving object 205 into multiple sub-objects. 210. Subobject 210 may be a license plate on a bus or a person's face.

[0052] Step 305 The surveillance camera 100 can determine that at least one of the sub-objects 210 extends outside the overlapping region 203.

[0053] Step 306 The surveillance camera 100 can subdivide at least one sub-object 210 extending outside the overlapping area 203 into further sub-objects 210, and the hard handover in step 309 below can be performed on the further sub-objects 210. For example, if the moving object 203 is a bus and the sub-object is a license plate, the further sub-objects could be each letter and / or digit on the license plate.

[0054] Step 307 The surveillance camera 100 performs a hard handover of each sub-object 210 from the first video stream 201a to the second video stream 201b when each sub-object 210 is completely within the overlap region 203 and when a point of each sub-object 210 passes through the boundary 208 within the overlap region 203. The hard handover is from the first video stream 201a, captured in one instance, to the second video stream 201b, captured in the next instance; in other words, the hard handover is between two frames captured in different time instances.

[0055] Each sub-object can be represented by image data from the first video stream 201a before the hard handover, and each sub-object 210 can be represented by image data from the second video stream 201b after the hard handover.

[0056] Before the hard handover, the seam 213 extends through the overlapping region 203, allowing it to adaptively surround the first end of each sub-object 210 as each sub-object 210 moves through the overlapping region 203. After the hard handover, the seam 213

[0057] As each sub-object 210 moves through the overlapping region 203, it can be moved to extend through the overlapping region 203 and adaptively surround the second end of each sub-object 210. The first end may be the end opposite the second end. Each sub-object 210 can be represented by image data from the first video stream 201a on the first side of the seam 213 before the hard handover. Each sub-object 210 can be represented by image data from the second video stream 201bona on the second side of the seam 213 after the hard handover.

[0058] Step 308 The surveillance camera 100 uses a blending algorithm to blend image data from the first video stream 201a and image data from the second video stream 201 with the image data of the sub-object 210 within the overlapping region 203 in a way that is unnatural.

[0059] In other words, the surveillance camera 100 can blend image data in the regions of the first video stream 201a and the second video stream 201b where the blending algorithm is enabled, i.e., in the regions where the blending algorithm is not disabled. The regions where the blending algorithm is enabled, i.e., not disabled, are regions containing image data that does not represent or constitutes a sub-object 210. Blending can be considered an optional step.

[0060] Step 309 The surveillance camera 100 can disable the blending algorithm for each of the sub-objects 210. In other words, the surveillance camera 100 can disable the blending algorithm in the regions of the first video stream 201a and the second video stream 201b representing the sub-objects 210, i.e., the regions containing the image data contained within the sub-objects 210. Therefore, it is not necessary to perform blending on the sub-objects 210.

[0061] The blending algorithm may be disabled by default for each of the sub-objects 210 and enabled for the remaining image data, or it may be determined each time the method is performed for each sub-object 210 or determined each time it is determined.

[0062] After at least some of the steps in Figure 3 have been performed, the first video stream 201a and the second video stream 201b can together provide a panoramic video stream.

[0063] To perform the method steps shown in Figure 3 for handling the first video stream 201a and the second video stream 201b, the surveillance camera 100 may include the arrangement shown in the figure. 4. In addition to the arrangement shown in the figure, the surveillance camera 100 includes a first image sensor 103a and a second image sensor 103b, as shown in the figure. 1.

[0064] The surveillance camera 100 is configured, for example, by an acquisition module 401 to acquire a first video stream 201a from a first image sensor 103a and a second video stream 201b from a second image sensor 103b. The field of view of the first video stream 201a partially overlaps with the field of view of the second video stream 201b within the overlap region 203. The first video stream 201a and the second video stream 201b can be obtained directly from their respective image sensors 103 or via some other entity, such as local memory or cloud memory configured within the surveillance camera 100. The acquisition module 401 may also be called an acquisition unit, acquisition means, acquisition circuit, acquisition means, etc. The acquisition module 401 may be the processor 403 of the surveillance camera 100, or may be configured within the processor 403 of the surveillance camera 100.

[0065] The surveillance camera 100 may be adapted, for example by a blending module 405, to blend image data from the first video stream 201a and the second video stream 201b, excluding the sub-object 210 in the overlapping region 203, using a blending algorithm. The default setting may be that the blending algorithm is enabled for image data other than the sub-object 210 and disabled for image data representing the sub-object 210. The blending module 405 is also called a blending unit, blending means, blending circuit, blending means, etc. The blending module 405 may be the processor 403 of the surveillance camera 100, or it may be composed of the processor 403 of the surveillance camera 100.

[0066] The surveillance camera 100 is configured to identify moving objects 205 within the overlapping area 203, for example, by an identification module 406. The moving objects may be, for example, a bus driving down the street or a person walking. The moving objects 205 may extend outside the overlapping area 203 in at least one direction. The identification module 406 may also be called an identification unit, identification means, identification circuit, identification means, etc. The identification module 406 may also be the following processor 403.

[0067] It is included in the surveillance camera 100 or the processor 403 of the surveillance camera 100.

[0068] The surveillance camera 100 may be adapted, for example, by an acquisition module 401 to obtain information indicating prioritized and unprioritized sub-objects.

[0069] The moving object 205 may belong to an object class. An object class can be associated with prioritized and unprioritized sub-objects. Note that the multiple sub-objects to which the moving object 205 is hard-handed over may be prioritized sub-objects. In the example where the bus is the moving object 205, the license plate may be a prioritized sub-object, while the driver's face may be an unprioritized sub-object.

[0070] The surveillance camera 100 is configured to divide a moving object 205 into multiple sub-objects 210, for example, by a division module 408. The sub-objects 210 may be a license plate on a bus or a person's face. The division module 408 may also be called a division unit, division means, division circuit, division means, etc. The division module 408 may be the processor 403 of the surveillance camera 100, or it may be composed of the processor 403 of the surveillance camera 100.

[0071] The surveillance camera 100 may be adapted, for example, by a decision module 410 to determine that at least one of the sub-objects 210 extends outside the overlapping area. 203. The decision module 410 may also be called a decision unit, decision means, decision circuit, decision means, etc. The decision module 410 may be the processor 403 of the surveillance camera 100, or may consist of the processor 403 of the surveillance camera 100.

[0072] The surveillance camera 100 may be configured, for example, by a splitting module 408 to subdivide at least one sub-object 210 extending outside the overlapping area 203 into further sub-objects 210, and the hard handover in step 309 below may be performed on the further sub-objects 210. For example, if the moving object 203 is a bus and the sub-object is a license plate, the further sub-objects could be each letter and / or number on the license plate. The surveillance camera 100 may be adapted, for example, by a disabling module 413 to disable the blending algorithm for image data representing each of the sub-objects.

[0073] 210. The disable module 413 is also called a disable unit, disable means, disable circuit, disable means, etc. The disable module 413 may be the processor 403 of the surveillance camera 100, or may consist of the processor 403 of the surveillance camera 100.

[0074] The surveillance camera 100 is configured, for example, by the execution module 415, to perform hard handovers of each sub-object 210 from the first video stream 201a to the second video stream 201b when each sub-object 210 is entirely within the overlap region 203 and when the point of each sub-object passes through the boundary 208 within the overlap region 203. The hard handover is from the first video stream 201a captured in one instance to the second video stream 201b captured in the next instance; that is, the hard handover is between two frames captured in different time instances. The execution module 415 may also be called an execution unit, execution means, execution circuit, execution means, etc. The execution module 415 may be the processor 403 of the surveillance camera 100, or it may be composed of the processor 403 of the surveillance camera 100.

[0075] Each sub-object 210 can be represented by image data from the first video stream 201a before the hard handover, and each sub-object 210 can be represented by image data from the second video stream 201b after the hard handover.

[0076] Before the hard handover, the seam 213 extends through the overlapping region 203, and can adaptively surround the first end of each sub-object 210 as each sub-object 210 moves through the overlapping region 203. After the hard handover, the seam 213 extends through the overlapping region 203, and can move to adaptively surround the second end of each sub-object 210 as each sub-object 210 moves through the overlapping region 203. The first end may be the end opposite to the second end. Each sub-object 210 can be represented by image data from a first video stream 201a on the first side of the seam 213 before the hard handover, and each sub-object 210 can be represented by image data from a second video stream 201b on the second side of the seam 213 after the hard handover.

[0077] Both the first video stream 201a and the second video stream 201b can provide a panoramic video stream.

[0078] The surveillance camera 100 includes a processor 403 and memory 418. The memory 418 contains instructions that can be executed by the processor 403.

[0079] The mechanism for handling the first video stream 201a and the second video stream 201b can be implemented via one or more processors, such as processor 403 in the arrangement shown in the figure, along with computer program code for performing the functions of this embodiment. The processor may be, for example, a digital signal processor (DSP), an application-specific integrated circuit (ASIC) processor, a field-programmable gate array (FPGA) processor, or a microprocessor. The program code may also be provided as a computer program product in the form of a data carrier that carries the computer program code for performing the embodiments herein, for example, when loaded into the surveillance camera 100. One such carrier may be in the form of a CD-ROM disk. However, other data carriers such as a memory stick are also possible. The computer program code may also be provided as pure program code on a server and downloaded to the surveillance camera 100.

[0080] A computer program, when executed on at least one processor, may include instructions that cause at least one processor to perform the method steps shown in Figure 3. The carrier wave may include the computer program, and the carrier wave may be an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0081] The term "at least one of A and B" should be understood to mean "A only, B only, or both A and B," where A and B refer to any parameters, numbers, designations, etc., as used herein.

[0082] It should be emphasized that, as used herein, the term “contains / includes” is taken to specify the presence of the described feature, integer, step, or component, but does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof. Furthermore, it should be noted that the words “a” or “an” preceding an element do not exclude the presence of multiple such elements.

[0083] As used herein, the term “configured” may also be translated as “arranged,” “adapted,” “possible,” or “operable.”

[0084] Furthermore, it should be emphasized that the steps of the methods defined in the appended claims may be performed in an order other than that in which they appear in the claims, without departing from the disclosure herein.

Claims

1. A method performed by a surveillance camera for processing a first video stream and a second video stream, the surveillance camera comprising a first image sensor and a second image sensor, the method comprising: acquiring a first video stream from a first image sensor and a second video stream from a second image sensor, a field of view of the first image sensor partially overlapping a field of view of the second image sensor in an overlap region; identifying a moving object within the overlap region; Dividing the moving object into a plurality of sub-objects; performing a hard handover of each of the plurality of sub-objects from the first video stream to the second video stream based on each of the plurality of sub-objects being entirely within the overlap region and based on a point of each of the plurality of sub-objects passing through a boundary within the overlap region, each respective sub-object of the plurality of sub-objects being represented by image data from the first video stream before the hard handover and each respective sub-object of the plurality of sub-objects being represented by image data from the second video stream after the hard handover; blending image data from the first video stream and image data from the second video stream other than image data for each of the plurality of sub-objects from the second video stream in overlap regions using a blending algorithm; disabling the blending algorithm for image data representing each of the plurality of sub-objects; determining that at least one of the plurality of sub-objects extends outside the overlap region; subdividing at least one of the plurality of sub-objects that extends outside the overlap region into a further sub-object, a hard handover is performed to said further sub-object. prior to the hard handover, a seam extends through the overlap region and adaptively surrounds a first end of each of the plurality of sub-objects based on each of the sub-objects from the first video stream moving through the overlap region; after the hard handover, the seam extends through the overlap region and is adaptively moved to surround a second end of each of the plurality of sub-objects based on each of the plurality of sub-objects moving through the overlap region; the first end is an end opposite the second end; each of the plurality of sub-objects is represented by image data from the first video stream on a first side of the seam before the hard handover; The method of claim 1 , wherein each of the plurality of sub-objects is represented by image data from the second video stream on a second side of the seam after the hard handover.

3. The moving object belongs to an object class, the object class is associated with a priority sub-object and a non-priority sub-object; The method according to claim 1 , wherein the sub-object among the plurality of sub-objects, into which the moving object is divided and into which a hard handover is performed, is the priority sub-object.

4. The method of claim 3, further comprising obtaining information indicating the prioritized sub-objects and the non-prioritized sub-objects.

5. The method of claim 1, wherein the blending algorithm is enabled as a default setting.

6. The method of claim 1, wherein the moving object extends outside the overlap region in at least one direction.

7. The method of claim 1, wherein the first video stream and the second video stream together provide a panoramic video stream.

8. A surveillance camera, A first image sensor; A second image sensor; and and a circuit, the circuit comprising: acquiring a first video stream from a first image sensor and a second video stream from a second image sensor; a field of view of the first image sensor partially overlaps a field of view of the second image sensor in an overlap region; the object is a moving object, identifying the moving object within the overlap region; Dividing the moving object into a plurality of sub-objects; performing a hard handover of each of the plurality of sub-objects from the first video stream to the second video stream based on each of the plurality of sub-objects being entirely within the overlap region and based on a point of each of the plurality of sub-objects passing through a boundary within the overlap region; each respective sub-object of the plurality of sub-objects is represented by image data from the first video stream before the hard handover, and each respective sub-object of the plurality of sub-objects is represented by image data from the second video stream after the hard handover; blending image data from the first video stream and image data from the second video stream other than image data for each of the plurality of sub-objects from the second video stream in overlap regions using a blending algorithm; Disabling the blending algorithm for image data representing each of the plurality of sub-objects; determining that at least one of the plurality of sub-objects extends outside the overlap region; subdividing at least one of the plurality of sub-objects that extends outside the overlap region into a further sub-object; The surveillance camera is configured such that a hard handover is performed to said further sub-object. prior to the hard handover, a seam extends through the overlap region and adaptively surrounds a first end of each of the plurality of sub-objects based on each of the sub-objects from the first video stream moving through the overlap region; after the hard handover, the seam extends through the overlap region and is adaptively moved to surround a second end of each of the plurality of sub-objects based on each of the plurality of sub-objects moving through the overlap region; the first end is an end opposite the second end; each of the plurality of sub-objects is represented by image data from the first video stream on a first side of the seam before the hard handover; The surveillance camera of claim 8 , wherein each of the plurality of sub-objects is represented by image data from the second video stream on a second side of the seam after the hard handover.

10. The moving object belongs to an object class, the object class is associated with a priority sub-object and a non-priority sub-object; The surveillance camera according to claim 8 , wherein among the plurality of sub-objects, a sub-object into which the moving object is divided and a hard handover is executed is the priority sub-object.

11. A surveillance camera as described in claim 8, wherein the circuit is further configured to obtain information indicative of the prioritized sub-objects and the non-prioritized sub-objects.

12. A surveillance camera as described in claim 8, wherein the blending algorithm is enabled as a default setting.

13. A surveillance camera as described in claim 8, wherein the moving object extends outside the overlap area in at least one direction.

14. The surveillance camera of claim 8, wherein the first video stream and the second video stream together provide a panoramic video stream.

15. A non-transitory computer-readable storage medium comprising computer-readable instructions that, when executed on at least one processor, cause the at least one processor to perform a method, the method comprising: acquiring a first video stream from a first image sensor and a second video stream from a second image sensor, a field of view of the first image sensor partially overlapping a field of view of the second image sensor in an overlap region; identifying a moving object within the overlap region; Dividing the moving object into a plurality of sub-objects; performing a hard handover of each of the plurality of sub-objects from the first video stream to the second video stream based on each of the plurality of sub-objects being entirely within the overlap region and based on a point of each of the plurality of sub-objects passing through a boundary within the overlap region, each respective sub-object of the plurality of sub-objects being represented by image data from the first video stream before the hard handover and each respective sub-object of the plurality of sub-objects being represented by image data from the second video stream after the hard handover; blending image data from the first video stream and image data from the second video stream other than image data for each of the plurality of sub-objects from the second video stream in overlap regions using a blending algorithm; disabling the blending algorithm for image data representing each of the plurality of sub-objects; determining that at least one of the plurality of sub-objects extends outside the overlap region; subdividing at least one of the plurality of sub-objects that extends outside the overlap region into a further sub-object, a hard handover is performed for the further sub-object.

Images