System and method for background modeling of video stream
Update the background model by analyzing image frame sequence changes on mobile cameras and using image segmentation and object detection algorithms to quickly reset when the camera stops, combining traditional and AI methods, the error classification of traditional background modeling on mobile cameras and Processing capacity limits, enabling robust and energy-efficient background modeling.
Patent Information
- Application Number
- CN202410636706.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-05
- Filing Date
- 2024-05-22
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-05-22
AI Technical Summary
For cameras with mobility capabilities, traditional motion-based background modeling is difficult to distinguish between camera movement and scene object movement, resulting in misclassification of background modeling and difficult to effectively update the background model with limited processing capabilities.
Update the background model by analyzing changes in the image frame sequence and quickly reset the background model using image segmentation and object detection algorithms when the camera stops moving, combining traditional and AI-based approaches for robust and energy-efficient background modeling.
It realizes the rapid generation of effective background models, reduces power peaks, adapts to the processing capacity limitations of mobile cameras, and improves the accuracy and efficiency of background modeling.
Smart Images

Figure CN119090913A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a method for background modeling of a video stream collected by a camera with mobile capability and further relates to an image processing system for implementing the method. Background Art
[0002] Video processing is a special case of signal processing. Video processing uses hardware, software, or a combination thereof to process a sequence of images representing a video. In video processing, a variety of processing tasks require preprocessing. A common step in such a process is to define a background model and apply it to the video sequence. Background modeling can be described as the division of an image frame into portions or regions that are classified as background or foreground. The resulting background and foreground segmentation can allow further video processing to focus on portions of the image frame that are relevant to certain processing tasks. For example, some processing tasks may only need to be performed in the foreground, while other processing tasks are only relevant to being performed in the background.
[0003] One type of background modeling is based on the analysis of motion in the video stream. If a part of the image does not change significantly, it can be considered to belong to the background. This type of background modeling is an affordable and effective way to provide a background model that is useful for a variety of applications. It can be used, for example, for background overlay applications and dynamic privacy masking applications. For fixed cameras, traditional background modeling based on motion in the video stream, used together with, for example, a configurable background merging time, can generally be considered a fairly effective solution.
[0004] However, background modeling is often more challenging for cameras with movement capabilities, such as pan-tilt-zoom (PTZ) cameras. A PTZ camera is a camera that can control direction and zoom. Since such cameras can move in response to rapid changes in the scene, traditional background modeling can be challenging in correctly identifying the background and foreground in a video stream by analyzing the changes, because background modeling based on motion analysis cannot distinguish between changes caused by the movement of the camera and changes caused by the movement of objects in the scene. Therefore, when using such modeling, there is a risk that some or all parts of the scene that do not move are incorrectly classified as foreground when the camera moves.
[0005] In addition to the above-mentioned challenges of background modeling for cameras with mobility capabilities, such cameras often have other requirements or limitations. For example, there may be limited processing power, especially when the camera is moving, which makes possible alternatives impossible or inappropriate. Therefore, there is a need for improved background modeling of video streams acquired by cameras with mobility capabilities. Summary of the invention
[0006] The present invention relates to a method for background modeling of a video stream collected by a camera with mobile capability, the method comprising:
[0007] Acquire a video stream comprising a sequence of image frames;
[0008] repeatedly updating a background model of the video stream by analyzing changes in a sequence of image frames and classifying image regions in the image frames that do not change over time as background;
[0009] Detecting camera movement and, optionally, stopping repeatedly updating the background model;
[0010] When it is detected that the camera movement stops or the camera movement is below a movement threshold, resetting the background model is performed by the following steps: acquiring one or more image frames; applying an image segmentation and / or object detection algorithm to the one or more image frames to identify at least a foreground object; classifying an area outside the identified foreground object as background;
[0011] After the resetting of the background model has been performed, it returns to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image regions in the image frames that do not change over time as background.
[0012] The inventors have realized that by combining two ways of background modeling for cameras with the described mobile capabilities, robust and energy-efficient background modeling can be achieved. By quickly resetting the background model when the camera stops moving, the background modeling can quickly obtain a valid background and begin further processing of image frames that require a background model. Preferably, the step of repeatedly updating the background model by analyzing changes in a sequence of image frames and classifying image areas in the image frames that do not change over time as background is a continuous process, and the resetting of the background model is a single event. The image segmentation and / or object detection algorithm used to perform the resetting of the background model can typically include the use of a neural network or other suitable artificial intelligence (AI)-based method. Although AI-based methods are good at determining background and foreground objects, these methods may also require more processing power. In the example of a mobile PTZ camera, the motor may cause significant power peaks during movement. In addition, when the camera is not moving, there are usually power limitations depending on how the camera is powered.
[0013] The inventors have discovered a hybrid solution that addresses the above challenges, in which traditional modeling is used to repeatedly update the background model of the video stream whenever there is no or limited motion, while image segmentation and / or object detection algorithms are used to perform a fast reset when the camera is stopped.
[0014] During fast movement, several options are available. One is to stop maintaining the background model during fast movement. In case of applications involving e.g. static background overlays, where the overlay based on static positions is only applied to background areas and not to foreground areas, it is possible to make the overlay unconditionally visible, i.e. visible to both background and foreground. In case of dynamic masking, full screen masking can be temporarily applied during fast movement. A single fast reset of the background can be performed to quickly restore the background as soon as the camera stops.
[0015] The present invention further relates to an image processing system comprising:
[0016] at least one camera, such as a pan-tilt-zoom camera; and
[0017] The processing unit is configured to:
[0018] Acquire a video stream comprising a sequence of image frames;
[0019] repeatedly updating a background model of a video stream by analyzing changes in a sequence of image frames and classifying image regions in the image frames that do not change over time as background;
[0020] Detecting camera movement and, optionally, stopping repeatedly updating the background model;
[0021] When it is detected that the camera movement stops or the camera movement is below the movement threshold, resetting the background model is performed by the following steps: acquiring one or more image frames; applying an image segmentation and / or object detection algorithm to the one or more image frames to identify a foreground object; classifying an area outside the identified foreground object as background;
[0022] After the resetting of the background model has been performed, it returns to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image regions in the image frames that do not change over time as background.
[0023] The system may further include a display for displaying a sequence of image frames of the video stream. Typically, further processing will also be applied to the displayed video stream, such as dynamic privacy masking, overlays, or any type of image / video processing. Dynamic privacy masking may include masking one or more foreground areas, i.e., moving objects of the scene. In contrast, static privacy masks are fixed-position masks, such as masking user-defined image areas in the scene. Fixed-position masks can be used to mask, for example, windows of a building.
[0024] The term "overlay" refers to added content or information, usually superimposed on an image frame. Overlays may be limited to covering background areas. Overlay content or information may include, but is not limited to, text, images, symbols, graphics, charts, statistics, boxes, etc. The image processing system may be used, for example, in camera-based surveillance systems.
[0025] Those skilled in the art will recognize that the presently disclosed method of background modeling for a video stream acquired by a camera having movement capabilities may be performed using any embodiment of the presently disclosed image processing system, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Various embodiments are described below with reference to the accompanying drawings. The drawings are examples of embodiments and are intended to illustrate some features of the presently disclosed method and system for background modeling of video streams and are not intended to limit the presently disclosed method and system.
[0027] Figure 1 A method flow chart of an embodiment of a background modeling method for a video stream according to the current disclosure is shown.
[0028] FIG. 2A to FIG. 2B is a schematic diagram of an image frame in a video sequence, the image frame having a background that defines whether a spatial region in the image frame belongs to the background or the foreground.
[0029] Figure 3 is a schematic diagram of an image frame in a video sequence where a foreground object transforms into a background object.
[0030] Figure 4 An example of image frames in a video sequence as the camera moves is shown.
[0031] Figure 5 An image frame is shown in which a plurality of objects have been detected and in which bounding boxes have been added.
[0032] Figure 6 An example of a conceptual framework for object detection using neural networks is shown.
[0033] Figure 7 A schematic diagram of an embodiment of the presently disclosed image processing system is shown. DETAILED DESCRIPTION
[0034] The present disclosure relates to a background modeling method for a video stream captured by a camera with mobile capability. Figure 1 A flow chart of a method according to an embodiment of the presently disclosed method 100 is shown, the method 100 being used for background modeling of a video stream acquired by a camera having mobile capabilities. The method 100 comprises the following steps:
[0035] Acquiring a video stream (101) comprising a sequence of image frames;
[0036] repeatedly updating a background model (102) of a video stream by analyzing changes in a sequence of image frames and classifying image regions in the image frames that do not change over time as background;
[0037] Detecting camera movement (103) and, optionally, stopping repeatedly updating the background model;
[0038] When it is detected that the camera movement stops or the camera movement is below a movement threshold, resetting the background model is performed by the following steps: acquiring one or more image frames; applying an image segmentation and / or object detection algorithm to the one or more image frames to identify at least a foreground object (104); classifying an area outside the identified foreground object as background (105);
[0039] After the resetting of the background model has been performed, the process returns to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image regions in the image frames that do not change over time as background (106).
[0040] "Background model" can generally be regarded as a term understood by those skilled in the art in the context of the present disclosure. In the context of the present disclosure, "background model" should be interpreted as a data model that determines, for each spatial region in a video sequence, whether the spatial region belongs to the background or foreground in the video sequence. The data model can store this information or process it in any suitable way. "Background" can include any area of an image frame whose image data is sufficiently similar to the corresponding area in the previous image frame, for example, in terms of pixel data. For example, the brightness and / or color intensity of pixels or groups of pixels between image frames in a video stream can be compared. Generally, the background is intended to correspond to an area in the monitoring scene that is not particularly relevant from an image analysis perspective. Similarly, "foreground" can include any area of an image frame whose image data is very dissimilar to the corresponding area in the previous image frame. Foreground objects are usually in motion. In a practical and simplified context, the foreground can correspond to a monitored object, such as a person, a car, cargo, etc. In the context of the present disclosure, "spatial region" can be interpreted as any number of pixels or sub-pixels in an image frame, which can be further grouped according to, for example, the shape of the object or the object to which it belongs.
[0041] The step of detecting camera movement may include detecting whether the camera movement is greater than a predefined limit. If there is only slow movement, traditional background modeling may still work fine.
[0042] The step of repeatedly updating the background model may be paused when camera movement is detected or the camera movement is greater than a predefined limit. When the camera moves quickly, it may stop maintaining the background model during the movement. An embodiment of the currently disclosed background modeling method further includes the following steps: when camera movement is detected, but before the camera movement stops or the camera movement is detected to be below a movement threshold, a background overlay covering the image area of the entire image frame is made visible and / or full masking and / or pixelation is applied to the image area covering the entire image frame. In other words, in the case of a background overlay, one option is to make the entire overlay visible. In the case of a dynamic privacy mask, a full screen mask can be switched to. Once the camera movement ends, a reset of the background model is performed, and the process can return to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image areas that do not change over time in the image frames as background.
[0043] The following steps may be sequential: repeatedly updating the background model; detecting camera movement; detecting that the camera movement stops or the camera movement is below a movement threshold; and performing a reset of the background model.
[0044] Repeatedly update the background model of the video stream
[0045] The step of repeatedly updating a background model of a video stream by analyzing changes in a sequence of image frames and classifying image regions in the image frames that do not change over time as background may include analyzing sub-regions of the sequence of image frames, wherein a sub-region is classified as a background sub-region if no significant changes occur between image frames in the sub-region during a predefined time period.
[0046] Figure 2A , Figure 2B and Figure 3 An embodiment of conventional background modeling of the presently disclosed method is shown.
[0047] Figure 2A An image frame 200 in a video sequence is shown. The video sequence includes a background model that defines whether a spatial region in the image frame 200 belongs to the background 201 or the foreground 202. In this embodiment, any significant change in the background 201 converts the corresponding spatial region to the foreground 202. A spatial region that includes image data that is substantially unchanged over a sufficiently long period of time may be converted to the background.
[0048] For example, the image frame 200 may be divided into a plurality of spatial regions 203, and each spatial region 203 may be estimated as a background or a foreground according to whether the spatial region changes. Figure 2B 204 is shown superimposed with a schematic diagram of an example of a first algorithm including a plurality of timers 204. Figure 2AEach timer 204 is associated with a spatial region 203 of a plurality of spatial regions 203 in the image frame 200. If, before a predefined time limit, image data of a spatial region 203 of an image frame 200 does not change significantly relative to image data of the spatial region 220 of a previous image frame 200, the spatial region 203 is defined as an idle region 201 in the background model.
[0049] If a significant change in image data of the spatial region 203 of the image frame 200 relative to image data of the spatial region 203 of the previous image frame 200 is detected, the spatial region 203 of the background 201 may be transitioned to the foreground 202 .
[0050] The timer 204 is one way to implement repeated updates of the background model that has the advantage of being simple and computationally efficient. It is easy to implement thresholds and / or manipulate the results compared to similar algorithms. This is in part because the timer 204 is a number and numbers are simple. For example, by simply adding seconds to the timer 204, the timer 204 will reach its threshold more quickly without changing the threshold. As another example, by resetting the timer 204 at a predictable period, the timer 204 can be prevented from reaching its threshold.
[0051] The timer 204 can further count up and / or down without significantly increasing the computational complexity. When it is determined that the idle area 201 will not be converted, the timer 204 associated with the idle area 201 can be reset or paused, or the time limit of the timer 204 associated with the idle area 201 can be increased. Manipulating the timer 204 is a simple and effective way to keep the idle area 201 as the foreground area 202 in the background model. When the timer 204 of the algorithm associated with the idle area 201 reaches an indication threshold below the predefined time limit, the idle area 201 can be indicated to be converted from the foreground 202 to the background 201.
[0052] Figure 3 An example of a foreground object switching to a background object is shown. In this example, the lower foreground object 202a becomes idle in the middle image frame 200b. Therefore, the spatial area 203 including the object 202a becomes the idle area 201, and when the object 202a is in motion, they are not in the leftmost image frame 200a. The idle area 201 of the middle image frame 200b is indicated as switching from the foreground to the background. In the rightmost image frame 200c, the object 202a has therefore been switched to the background. The spatial area 203 is described in the figure as nine areas per image frame 200a-c. In fact, it can be more or less than described. For example, each pixel of the image frame 200a-b or each square of four pixels can be a spatial area 203.
[0053] As known to those skilled in the art, Figure 2A , Figure 2B and Figure 3 The embodiments described in are not the only way to repeatedly update a background model of a video stream by analyzing changes in a sequence of image frames and classifying image regions in the image frames that do not change over time as background.
[0054] Reset background model
[0055] As described above, by using a fast reset of the background model when the camera stops moving, background modeling can quickly generate a valid background and begin further processing of the image frame, which may require a background model.
[0056] Preferably, the step of performing a resetting of the background model is performed substantially immediately after detecting that the camera movement has stopped or that the camera movement has fallen below a movement threshold. There are several methods for determining the camera movement and the extent of the camera movement, which are readily available to a person skilled in the art. A straightforward method is to use a control signal controlling the camera movement to achieve this, but it is also possible to use, for example, physical data from measurements of the camera or by analyzing the captured images, for example by studying how reference points in the images move.
[0057] The conventional repetitive step of updating the background model is typically a continuous process. As previously described, in this process, the sequence of image frames is analyzed for changes over time. In contrast, the step of resetting the background model can be a single event that is preferably fast. Preferably, the step of applying the image segmentation and / or object detection algorithm does not involve an analysis of changes over time, but is performed on a limited number of images, or even on a single image. When the camera movement stops, it may be useful to generate a fast first background model, rather than using conventional background modeling, which typically requires some time to generate the background model. It may be noted that a short power peak is acceptable at this point because the movement that would otherwise use a lot of power has stopped, or at least is relatively low. Once the rapid resetting of the background model is completed, the process can return to the process of repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image areas in the image frames that do not change over time as background.
[0058] Image segmentation can be, for example, object segmentation, instance segmentation, semantic segmentation, or panoptic segmentation. All of these examples of image segmentation do not include analysis of motion over time, i.e., changes between several image frames, to classify image regions that do not change over time in image frames as background. Therefore, they can be applied immediately to one or a limited number of image frames. The advantages of resetting the background model described in the present disclosure can be illustrated by the following example. The frame rate of the camera can be, for example, 30 frames per second (fps), 50fps, 60fps, or 120fps. A traditional algorithm that updates the background model by analyzing changes in a sequence of image frames and classifying image regions that do not change over time in image frames as background may take, for example, 2 seconds. In other words, when the camera movement stops or the camera movement is below a movement threshold, it may take several seconds until traditional background modeling is able to generate the expected background model. The background model reset described in the present disclosure can be performed significantly faster, for example, in less than 300 milliseconds, preferably in less than 200 milliseconds, and even more preferably in less than 100 milliseconds. Therefore, image segmentation and / or object detection algorithms can be applied to a single image frame, which will provide a faster background model than the background model that traditional background modeling can generate. The image segmentation and / or object detection algorithm can be applied to additional image frames, such as less than 3 image frames, less than 5 image frames, or less than 8 image frames. In principle, the image segmentation and / or object detection algorithm can be repeatedly used until traditional background modeling is ready to produce a background model. In one embodiment of the currently disclosed method for background modeling of a video stream acquired by a camera with mobile capabilities, the step of applying image segmentation to one or more image frames to identify at least a foreground object includes performing image segmentation on a single image frame or on multiple independent image frames without analyzing motion to identify foreground and background.
[0059] In the context of this disclosure, the term "object detection" refers to detecting one or more objects in an image and returning their locations, usually in the form of coordinates. In some of the most common object detection applications, a bounding box indicates the detected object.
[0060] In the present disclosure, image segmentation and / or object detection algorithms are applied to one or more image frames to identify at least foreground objects. The method can be trained to know what type of objects will typically be foreground objects. This can typically be objects that are known to be able to move in the video stream, such as people, vehicles, animals, etc. The method can additionally have knowledge about which objects will be considered background. This can typically be objects that are known to be stationary in the video stream, such as buildings.
[0061] The step of identifying at least the foreground object by image segmentation and / or object detection algorithms may include applying a machine learning model, such as a neural network, trained to recognize objects, such as the above-mentioned objects. The neural network may include, for example, a deep learning model. There are a variety of neural network algorithms / techniques that can be used to detect objects, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc. In general, those skilled in the art are able to perform such object detection with the aid of neural networks.
[0062] According to a non-limiting example of a setting for object detection using a neural network, the neural network is input with labeled data. The labeled data is, for example, an image of an object, wherein the image is labeled with the correct type of object, i.e., the labeled data includes the image data and the ground truth of the image data itself. The image data is input to a classifier, and the ground truth is sent to a loss function calculator. The classifier processes the data representing the object to be classified and generates a classification identifier. The processing in the classifier may include applying weights to the values as the data passes through the classifier. The classification identifier may be a feature vector, a classification vector, or a single value identifying a category. In the loss function, the classification identifier is compared with the ground truth using, for example, a loss function. The result of the loss function is then transmitted to a weight adjustment function, which is configured to adjust the weights used in the classifier. The process can then scan the image using a neural network to detect and classify objects in the image.
[0063] Figure 6 An example of a conceptual framework 300 for object detection using a neural network is shown. Some building blocks of a CNN are convolutional layers, pooling layers, flattening layers, and fully connected layers. The input to a CNN is a tensor. The convolutional layer convolves the input and passes the result to the next layer. This is similar to the response of neurons in the visual cortex to a specific stimulus. The convolutional network may further include a pooling layer. The pooling layer reduces the dimensionality of the data by combining the output of a cluster of neurons in one layer into a single neuron in the next layer. The CNN may further include a fully connected layer that connects each neuron in one layer to each neuron in another layer. Therefore, the flattened matrix can be passed through the fully connected layer to classify the image.
[0064] There are other object detection algorithms that do not rely on CNNs or machine learning.
[0065] In one embodiment of the currently disclosed method for background modeling of a video stream captured by a camera with mobility capabilities, an algorithm for performing image segmentation and / or object detection on one or more image frames to identify foreground objects includes thresholding and / or clustering and / or histogram-based segmentation and / or edge detection.
[0066] An example of an object detection algorithm is the Viola-Jones detection framework. In this method, an image frame is scanned with a sliding window, where each region is classified as containing or not containing an object. This method uses Haar features and cascade classifiers to detect objects. Various types of such object detection methods are known in the prior art, where cascaded identifiers are used to detect objects, such as described in Viola, Paul and Michael Jones, "Rapid Object Detection Using Enhanced Cascades of Simple Features", Computer Vision and Pattern Recognition, 2001, CVPR 2001, 2001 IEEE Computer Society Conference Proceedings, Volume I, IEEE 2001. Because visual features are important for these algorithms, groups of objects that share similar visual features can be detected, examples of such groups are faces, vehicles, people, etc. Any of these methods can be used alone or in combination to detect objects in image data. It is also possible to detect several objects in the same set of image data. When an object is detected, a set of identification features can be created to describe the visual appearance of the detected object.
[0067] Figure 5 An example of an image frame 200 is shown in which a plurality of objects 208 have been detected and in which a bounding box 207 has been added. Figure 5 In the example of FIG. 2 , object 208 is a car, but the object may be any item that is considered to belong to the foreground or background.
[0068] Figure 4 An example of image frames 200a / 200b in a video sequence when the camera moves is shown. In this example of camera movement, it can be seen that there is a camera pan between the left and right image frames. The fixed background objects (trees 205 and buildings 207) do not have the same position in the two image frames 200a and 200b. At the same time, there is a moving object 206, which has a different position relative to the background objects 205 and 207 in the two image frames 200a and 200b.
[0069] Figure 7 A schematic diagram of an embodiment of the presently disclosed image processing system 400 is shown. The disclosed image processing system 400 comprises a camera 401 , a display 403 and a processing unit 402 .
[0070] The system may (but need not) include a display for displaying the sequence of image frames. Background modeling may be useful in a variety of applications visualized on a display, including overlay and dynamic masking applications. However, in further applications, the background model is not necessarily displayed, but rather used in additional applications. Additional applications may include, for example, extracting statistics or further analysis of the video.
[0071] The system may further include peripheral components, such as one or more memories, which can be used to store instructions executable by the processing unit. The system may further include any of the following: internal and external network interfaces, input and / or output ports, keyboard or mouse, etc.
[0072] As will be appreciated by those skilled in the art, the processing unit may also be a single processor in a multi-core / multi-processor system.Both the computing hardware accelerator and the central processing unit may be connected to a data communications infrastructure.
[0073] The system may include a memory, such as a random access memory (RAM) and / or a read-only memory (ROM) or any suitable type of memory. The system may further include a communication interface that allows software and / or data to be transmitted between the system and an external device. The software and / or data transmitted through the communication interface may be any suitable form of electrical, optical or radio frequency signals. The communication interface may include, for example, a cable or a wireless interface.
[0074] The invention further relates to a computer program having instructions which, when executed by a computing device or a computing system, cause the computing device or the computing system to perform any embodiment of the presently disclosed method for background modeling of a video stream. The computer program may be stored on any suitable type of storage medium, such as a non-transitory storage medium.
Claims
1. A method for background modeling of a video stream, wherein the video stream is acquired by a camera having a mobile capability, wherein the method include: Acquire a video stream comprising a sequence of image frames; repeatedly updating a background model of the video stream by analyzing changes in the sequence of image frames and classifying image regions in the image frames that do not change over time as background; detecting camera movement and, optionally, ceasing to repeatedly update the background model; When it is detected that the camera movement stops or the camera movement is below a movement threshold, resetting the background model is performed by the following steps: acquiring one or more image frames; applying an image segmentation and / or object detection algorithm to the one or more image frames to identify at least a foreground object; classifying an area outside the identified foreground object as background; After the resetting of the background model has been performed, returning to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image areas in the image frames that do not change over time as background.
2. The method for background modeling of a video stream according to claim 1, in, During the step of performing the resetting of the background model, the step of repeatedly updating the background model is suspended.
3. The method for background modeling of a video stream according to claim 1, in, The step of performing said resetting of said background model is performed substantially immediately after detecting that said camera movement has stopped or that said camera movement has fallen below a movement threshold.
4. The method for background modeling of a video stream according to claim 1, in, The camera movement comprises pan and / or tilt and / or zoom.
5. The method for background modeling of a video stream according to claim 1, further comprising the step of performing background superposition and / or shielding based on the background model.
6. The method for background modeling of a video stream according to claim 1, further comprising: The following steps are involved: When camera motion is detected, but before camera motion is detected to have stopped or the camera motion is below a motion threshold, a background overlay is made visible to and / or full masking and / or pixelation is applied to an image area covering the entire image frame.
7. The method for background modeling of a video stream according to claim 1, in, The step of repeatedly updating the background model is a continuous process, and wherein said resetting of said background model is a single event.
8. The method for background modeling of a video stream according to claim 1, in, The step of applying an image segmentation and / or object detection algorithm to the one or more image frames to identify foreground objects is performed on a limited number of image frames.
9. The method for background modeling of a video stream according to claim 1, in, The step of applying image segmentation and / or object detection algorithms does not involve analysis of changes over time.
10. The method for background modeling of a video stream according to claim 1, in, The step of repeatedly updating the background model comprises analyzing a sub-region of the sequence of image frames, wherein a sub-region is classified as a background sub-region if no significant changes between the image frames occur in the sub-region during a predefined time period.
11. The method for background modeling of a video stream according to claim 1, in, The step of applying an image segmentation and / or object detection algorithm to the one or more image frames to identify foreground objects may comprise thresholding and / or clustering and / or histogram based segmentation and / or edge detection.
12. The method for background modeling of a video stream according to claim 1, in, The following steps are sequential: repeatedly updating a background model; detecting camera movement; detecting that camera movement stops or the camera movement is below a movement threshold; and performing a reset of the background model.
13. The method for background modeling of a video stream according to claim 1, further comprising displaying the sequence of image frames on a display.
14. A computer program having instructions, which, when executed by a computing device or a computing system, cause the computing device or the computing system to perform the method for background modeling of a video stream according to claim 1.
15. An image processing system, include: at least one camera, such as a pan-tilt-zoom camera; as well as The processing unit is configured to: Acquire a video stream comprising a sequence of image frames; repeatedly updating a background model of the video stream by analyzing changes in the sequence of image frames and classifying image regions in the image frames that do not change over time as background; detecting camera movement and, optionally, ceasing to repeatedly update the background model; When it is detected that the camera movement stops or the camera movement is below a movement threshold, the resetting of the background model is performed by the following steps: acquiring one or more image frames; applying an image segmentation and / or object detection algorithm to the one or more image frames to identify a foreground object; classifying an area outside the identified foreground object as background; After the resetting of the background model has been performed, returning to repeatedly updating the background model of the video stream by analyzing changes in the sequence of image frames and classifying image areas in the image frames that do not change over time as background.
Citation Information
Patent Citations
System and method for managing moving surveillance cameras
CN101272483A
Image segmentation for a live camera feed
CN105321171A
System and method for training object classifier by machine learning
CN109154976A
Image processing device, image processing method, and program
JP2021193505A
Device and Method for Camera Driven Background Subtraction
US20170116750A1
