Method, system and non-transitory computer-readable storage medium for detecting objects of a first object type in a video sequence

Abstract

To detect an object of a first object type in a video sequence.SOLUTION: A method analyzes each area of a scene in each image frame in a video sequence by a first algorithm for detecting a first object type, detects an object of the first object type in the last image frame among a plurality of image frames, provides a background model of the video sequence by using a second algorithm, identifies whether image data corresponding to the area has an uncertain object type status on the basis of an analysis by the first algorithm about each area of the scene, checks the background model in order to determine an object type status of an area of a first image in the case that the area has the uncertain object type status, depends on the analysis of the area of the first image to be performed by the first algorithm in the case of having no uncertain object type status, and masks or highlights image data determined to be a portion of the object of the first object type in the first image.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to object detection, and in particular to a method, system and non-transitory computer-readable storage medium for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene. [Background technology]

[0002] Object detection in video surveillance systems has evolved over the years as an essential component for a variety of applications, including security, traffic monitoring, retail analytics, and smart cities. The primary goal of these systems is to identify and track objects of interest, such as people, vehicles, or animals, in a video stream.

[0003] Object classification is often an essential component of an object detection system. Object detection involves identifying objects of interest in an image or video frame and assigning them to a specific category or class. The combination of object localization (finding the location of an object) and object classification (identifying the class of an object) enables a complete object detection system.

[0004] One problem with object detection systems is incorrectly classified objects, i.e., false positives or false negatives. Such classifications can, for example, negatively impact the user experience. In systems where a human operator reviews or interacts with the video output, incorrectly classified objects can result in confusion, fatigue, and frustration. This can reduce user confidence in the system and cause the operator to miss genuine objects of interest due to an increased number of false alarms.

[0005] Therefore, improvements in this context are needed. Summary of the Invention

[0006] In view of the above, it would be advantageous to overcome or at least mitigate one or more of the above-described disadvantages as set out in the attached independent patent claims.

[0007] According to a first aspect of the present invention, there is provided a method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the method comprising: for each image frame of the plurality of image frames, analysing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates, for each region of a plurality of regions in the scene, a probability that image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether the image data is part of an object of the first object type, and storing an indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list of indicators associated with the region.

[0008] The method further includes detecting an object of the first object type in a first image frame, the first image frame being a last image frame of the plurality of image frames, the detecting including providing a background model of the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data in the first image frame corresponding to the region belongs to the background or the foreground, the background model being defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames; identifying, for each region of the scene, whether image data corresponding to the region in the first image has an uncertain object type status based on an associated list of indicators; and upon identifying that the image data has an uncertain object type status, determining that the image data is not part of an object of the first object type when the background model defines the image data as belonging to the background in the first image; and determining that the image data is part of an object of the first object type when the background model defines the image data as belonging to the foreground in the first image.

[0009] In object classification applications designed to detect specific object types, problems can arise when object classes alternate between frames. Some objects may have appearances similar to the target object type, causing the classification algorithm to generate varying probabilities. These probabilities may exceed a threshold to classify the object as the target type in one frame, and then fall below the threshold to classify the object as a different type in the next frame. Factors contributing to these variations may include the object's viewing angle, image sensor noise, small changes in segmentation mask determination, or variations in the object's appearance.

[0010] Such inconsistent classification can lead to confusion for operators monitoring the output and can adversely affect system performance, decision-making, and privacy, among other problems.

[0011] Advantageously, using two different algorithms to determine object type can increase the robustness of the method. The second algorithm, which provides a background model of the video sequence, can be considered a backup in case the first algorithm outputs different assessments (probabilities above and below a probability threshold) for the corresponding image content across multiple image frames. Such varying outputs between multiple image frames for a particular region of the scene can lead to the region of the scene depicted in the currently processed image frame (referred to as the first image frame, which is the most recent image frame in the video sequence) acquiring an "uncertain object type status." By using a previous frame in the video sequence to determine whether the region of the scene depicted in the first frame (i.e., the image data depicting the region in the first frame) should acquire an uncertain object type status, prediction of future states is avoided, thus resulting in a low-complexity, robust method.

[0012] It is important to clarify that the term "first image frame" as used herein does not refer to the order of frames in a video sequence. Rather, it is simply a means of distinguishing one frame from another for purposes of describing or claiming the invention. The order of frames in a sequence is determined by their temporal relationship to one another, with the first frame being captured or displayed before the second frame, and so on until the last frame is reached.

[0013] By the term "uncertain object type status", in the context of this specification, it should be understood that the output from the first algorithm results in a mixture of classification results during analysis of multiple image frames, for example where an area of ​​the scene depicted in some of the image frames is classified as not being part of an object of the first object type, while in other of the image frames the area is classified as being part of an object of the first object type.

[0014] The degree of mixing that results in an uncertain object type status, e.g., the ratio between positive and negative classifications, the magnitude of the difference between the probability that the image data is part of an object of the first object type and a probability threshold, etc., depends on the use case and application requirements.

[0015] When an uncertain object type status is determined for particular image data of a first image, the background model is used to determine whether an object of the first object type has been detected in the image data.

[0016] Advantageously, in the method, false positive identification of a static object having an appearance similar to that of an object of the first object type can be avoided because the background model can determine that the image data depicting the static object in the first image frame belongs to the background. Furthermore, the background model can prevent false negative identification of a moving object belonging to the first object type even when their appearance in one or more of the image frames does not match the assessment criteria of the first algorithm. This is because the background model can determine that the image data representing the moving object in the first image frame belongs to the foreground, thus avoiding false negative identification.

[0017] Furthermore, the list of indicators may be continuously updated as new image frames are added to the video sequence, potentially using a first-in, first-out (FIFO) approach, so that the method can effectively adapt to new objects or other scene changes captured within the video sequence.

[0018] In some embodiments, the method further includes, upon identifying the image data as not having uncertain object type status, determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type, and determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type.

[0019] In other words, if the image data in the first image is not identified as having an uncertain object type status, the output from the first algorithm for the image data is trusted and the image data is determined to be part of an object of the first object type if the first algorithm made this assessment. Advantageously, a consistent assessment from the first algorithm can be relied upon and false positives and false negatives can be avoided.

[0020] In some examples, identifying whether the image data has an uncertain object type status in the first image includes determining whether an absolute value of a difference between the probability that the image data is part of an object of the first object type and a probability threshold based on the associated list of indicators exceeds a threshold difference; if the difference exceeds the threshold difference, determining that the image data does not have an uncertain object type status; if the absolute value of the difference does not exceed the threshold distance, determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences; if the distribution measure indicates a mix of positive and negative differences that are within an uncertainty range, determining that the image data has an uncertain object type status; otherwise determining that the image data does not have an uncertain object type status.

[0021] Advantageously, if the first algorithm is sufficiently certain in its assessment of the image data of the first image (the difference between the probability that the image data in the first image is part of an object of the first object type and the probability threshold exceeds the threshold difference), this assessment is relied upon and the image data is not considered to have an uncertain object type status. This embodiment may reduce the complexity of the method because if the first algorithm assesses a sufficiently high or sufficiently low probability that the image data of the first image is part of an object of the first object type, then examination of the remaining list of indicators (i.e., the assessment made for the previous frame) and checking of the background model are not required. Furthermore, false positives and false negatives may be avoided.

[0022] The threshold difference may depend on the requirements of the application in which the method is implemented and / or the reliability metric of the first algorithm. The threshold difference may be set to, for example, 0.3 on a scale of 0 to 1. If the probability threshold is 0.5, this means that a probability of 0.8 or greater, or 0.2 or less, is considered "sufficiently high" or "sufficiently low," respectively. Other threshold differences are equally possible and depend on the use case.

[0023] If the first algorithm is not sufficiently certain, the list of indicators is analyzed to determine whether the image data should be considered to have an uncertain object type status. The distribution between positive and negative differences is analyzed using any suitable statistical method, such as determining the ratio between negative and positive values, the mean of the list, the median of the list, the sum of the values ​​in the list, etc. If the distribution measure falls within the uncertainty range, the image data is considered to have an uncertain object type status; otherwise, the image data is not considered to have an uncertain object type status.

[0024] Because the list indicates positive and negative differences and the first algorithm may classify the image data into two classes (first object type or non-first object type), the uncertainty range typically includes a low threshold and an high threshold. For example, if a ratio is determined, the uncertainty range may indicate a mix of indicators indicating positive differences and indicators indicating negative differences between a 20 / 80 distribution and an 80 / 20 distribution. This means that the image data is considered uncertain if the ratio is between 20 / 80 and 80 / 20 (20 / 80, 30 / 70, 50 / 50, 60 / 40, 80 / 20, etc., but not 100 / 0, 0 / 100, 10 / 90, or 95 / 5). Any other suitable ratio for the range threshold, such as 10 / 90, 25 / 75, etc., may be used. If the mean value is used, the uncertainty range may include a lower value of -0.2 and a higher value of 0.3, so a mean value between -0.2 and +0.3 would be considered uncertain, but not values ​​outside that range (-0.3, +0.35, etc.).

[0025] In some embodiments, the step of identifying whether the image data has uncertain object type status in the first image includes determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences, and determining that the image data has uncertain object type status if the distribution measure indicates a mix of positive and negative differences that fall within the uncertainty range, and otherwise determining that the image data does not have uncertain object type status.

[0026] In this embodiment, the list of indicators is constantly analyzed, which may provide a more reliable assessment of the image data. Furthermore, this embodiment allows the list of indicators to be a list of binary values, with positive differences indicated by a first value of the binary value and negative differences indicated by a second value of the binary value. This may reduce the complexity of the method for both storing the list of indicators and determining the distribution measure of the list.

[0027] In some embodiments, the first algorithm includes a feature extraction process for extracting features of objects in an image and representing them as a vector of numbers. This means that the first algorithm may include an artificial intelligence (AI) or machine learning (ML) algorithm trained to detect objects of the first object type in an image. AI / ML is a suitable technique for classifying objects in images and can be relatively easily trained on large datasets of images labeled with objects of interest. Suitable AI / ML algorithms include Haar cascades, histograms of oriented gradients (HOG), local binary pattern histograms (LBPH), convolutional neural networks (CNNs), and transformer-type models (which take sequential data, such as video streams, as input).

[0028] In some embodiments, the second algorithm is a motion-based background detection algorithm. Examples of such algorithms include using a time-average filter, a frame differencing method, a mean filter, a running Gaussian mean, and a background mixture model (such as a Gaussian mixture model (GMM)). Other possible algorithms include an adaptive multi-band binary (AMBB) algorithm and an optical flow method. Using a motion-based background detection algorithm may provide a good backup algorithm if the first algorithm produces uncertain results, as described above.

[0029] In some embodiments, the list of region-related indicators is a FIFO list having 5 to 15 values. In other embodiments, the FIFO list may have more values, such as 20, 30, etc. Using a FIFO data structure may provide a simple, efficient, and effective method for calculating the distribution measures described herein, especially in real-time applications where new data points are being continuously added.

[0030] In some embodiments, the method further includes masking or highlighting, in the first image, image data determined to be part of an object of the first object type. When dealing with these types of applications, it can be frustrating for an operator if a particular object is masked or highlighted in one frame but not in the next. This inconsistency can be annoying and draw unnecessary attention. Furthermore, in masking applications, there are potential privacy concerns if an object is only occasionally masked. The present method may address or at least reduce one or more of these shortcomings.

[0031] In some embodiments, the method is implemented in a camera and the video sequence is part of a live video stream captured by the camera.

[0032] According to a second aspect of the present invention, there is provided a method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames, the method comprising: for each image frame of the plurality of image frames, analysing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates, for each object of the plurality of objects tracked in the image frame, a probability that the object is an object of the first object type, the first algorithm having a probability threshold for determining whether the object is an object of the first object type, and storing an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list of indicators associated with the object.

[0033] The method further includes detecting an object of the first object type in a first image frame, where the first image frame is the last image frame of the plurality of image frames, and the detecting includes providing a background model of the video sequence, where the background model defines, for each region of a plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground, and the background model is defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames.

[0034] The method further includes, for each object of the plurality of objects tracked in the first image, identifying whether the object has an uncertain object type status based on a list of indicators associated with the object; determining image data corresponding to the object in the first image if the object has an uncertain object type status; determining that the object is not an object of the first object type if the background model defines the image data as belonging to a background in the first image; and determining that the object is an object of the first object type if the background model defines the image data as belonging to a foreground in the first image.

[0035] The method of the second embodiment is similar to the method of the first embodiment, and can achieve the same or corresponding advantages. The difference is that in the second embodiment, the assessment from the first algorithm relates to objects detected in the video stream rather than to regions of the scene. Advantageously, the method of the second embodiment can handle both stationary and moving objects. As in the first embodiment, a background model is used to assess whether a tracked object with an uncertain object type status should be considered background or foreground in the first image frame. If the image data corresponding to the object should be considered foreground, the object is determined to be of the first object type; otherwise, the object is not determined to be of the first object type.

[0036] In some embodiments, each of the tracked objects is located beyond a threshold distance from the camera that captured the video sequence of the scene. For example, the first algorithm may have an effective range or detection range within which the first algorithm can accurately classify objects based on their features or characteristics in the image. This range is typically determined by the resolution and quality of the camera and the performance of the object classification algorithm. For objects outside the detection range, factors such as poor image quality, occlusion, and changes in lighting conditions may reduce classification accuracy. In embodiments, the threshold distance may be the same as or close to the detection range of the first algorithm. Using this embodiment, objects that may be classified as a first object type and are not considered background may be determined to be of the first object type. As a result, false negatives may be reduced. This may be particularly important in scenarios where objects of the first object type should be masked, for example, for privacy reasons.

[0037] The second embodiment may generally have the same features and advantages as the first embodiment.

[0038] According to a third aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing instructions for, when executed on a device having processing capability, performing a method according to the first or second aspect.

[0039] According to a fourth aspect of the present invention, there is provided a system for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the system comprising one or more processors and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to perform the method of the first or second aspect.

[0040] The third and fourth aspects may generally have the same features and advantages as the first aspect.

[0041] Other objects, features and advantages of the present invention will become apparent from the following detailed disclosure, the attached dependent claims and the drawings.

[0042] In general, all terms used in the claims should be interpreted according to their ordinary meaning in the art unless expressly defined otherwise herein. All references to "a / an / the [element, device, component, means, step, etc.]" should be openly interpreted as referring to at least one instance of the element, device, component, means, step, etc., unless expressly stated otherwise.

[0043] The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

[0044] The above, as well as additional objects, features, and advantages of the present invention will be better understood from the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the accompanying drawings, in which like reference numerals are used for similar elements, and in which: [Brief explanation of the drawings]

[0045] [Figure 1] FIG. 2 illustrates a video sequence including a plurality of images in which objects of a first object type are identified by a first algorithm according to a first embodiment. [Figure 2] FIG. 2 illustrates a video sequence including a plurality of images in which objects of a first object type are identified by a first algorithm according to a second embodiment. [Figure 3] FIG. 2 shows a background model of the video sequence in FIG. [Figure 4] FIG. 3 shows a background model of the video sequence in FIG. 2. [Figure 5] FIG. 10 illustrates a list of indicators, according to an embodiment. [Figure 6] FIG. 10 illustrates a list of indicators, according to an embodiment. [Figure 7] 1 is a flowchart of a method for detecting an object of a first object type in a first image frame of a plurality of image frames, according to an embodiment. [Figure 8] 1 is a flowchart of a method for detecting an object of a first object type in a first image frame of a plurality of image frames, according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0046] Object classification systems may output false positives or false negatives for a variety of reasons, including imbalanced training data, similar object appearance, occlusion, lighting differences, etc. For example, objects with similar visual features or appearances may confuse the classification system, resulting in false positives or false negatives. Models may struggle to distinguish between objects with similar shapes, textures, or colors, especially under certain lighting conditions or from certain viewpoints.

[0047] In many object classification systems, especially those in which the output is monitored by an operator, consistent false positives or false negatives may be ignored by the operator and may not attract much attention. However, inconsistent false positives or false negatives are more difficult to ignore and may lead to user frustration, reduced trust in the system, and reduced adoption of the technology. Furthermore, in real-time applications such as surveillance applications, inconsistent false positives or false negatives may result in unstable decision-making as the system constantly re-evaluates the presence or absence of objects of a particular object type. This may lead to inefficient or potentially dangerous decisions.

[0048] FIG. 1 illustrates an example of a video sequence in which an object classification algorithm outputs a variation classification for an object. FIG. 1 shows three images 100a-c from a video sequence capturing a scene. The scene includes four objects: a tree 108, a dog 104, a car 106 traveling on a road, and a person 102. Images 100a-c are further enhanced with highlighting features 110, 112, 114, and 116 that indicate image regions (image data) where the first algorithm identified an object of a first object type in each of image frames 100a-c. In the example of FIG. 1, the first algorithm is configured to detect human faces. This detection can serve various purposes, such as monitoring restricted areas where access is prohibited and alerting an operator when an individual enters such areas, which can also trigger an alert. Furthermore, face detection can be employed for privacy purposes, allowing for the masking or blurring of faces to protect the identity of individuals. It should be noted that the first algorithm may be configured to detect other object types, such as cars, animals, etc., depending on the use case of the object classification system.

[0049] The first algorithm may be configured to identify objects of a first object type in the image frames 100a-c by calculating, for each of a plurality of regions in a scene captured by the image frames 100a-c, a probability that image data corresponding to the region is part of an object of the first object type in the image frames 100a-c. The first algorithm may have a probability threshold for determining whether the image data is part of an object of the first object type. The first algorithm may be a machine learning or artificial intelligence model that offers various technical advantages, such as improved accuracy, adaptability to new data, the ability to handle complex relationships, and scalability. Typically, such an algorithm, or other suitable algorithm, classifies the object type of an object with a determined probability, for example, 0.7 or 0.2 on a scale of 0 to 1. If the determined probability exceeds a probability threshold, for example, 0.5, the object is determined to be of the corresponding object type; otherwise, the object is determined not to be of the corresponding object type. The probability threshold may be fixed for all images or may vary based on, for example, the number of objects in the image, the quality of the image, the lighting conditions of the image, etc.

[0050] The first algorithm may include a feature extraction process to extract features of objects in the image frames 100a-c and represent them as vectors of numbers. The vectors of numbers (feature vectors) may then be used to determine the probability that the image data of the object is part of an object of the first object type in the image frames 100a-c. The object's location may be mapped to regions of the scene. If the camera that captured the images is a static camera (as in the example of FIG. 1), each region of the scene may be represented by the same pixel index or pixel index range in each image frame 100a-c. In the case of a moving camera, motion data may be used in addition to the pixel index to determine corresponding pixel regions between the image frames 100a-c. If the image frames 100a-c were captured at different resolutions, a scaling parameter may be further used to determine corresponding pixel regions between the image frames 100a-c.

[0051] However, no matter how well the first algorithm is trained, the model may struggle to distinguish between objects with similar shapes, textures, or colors, especially under certain lighting conditions or from certain viewpoints. This is illustrated in Figure 1, where image data 114 depicting part of a leaf on tree 108 is interpreted as a human face in the last image 100c of tree images 100a-c. Apart from this incorrect classification, the first algorithm correctly interprets other areas of the scene as depicted in the three images 100a-c, classifying only the image data 110, 112, 116 corresponding to the area where person 102 is present in the scene as being part of an object of the first object type.

[0052] If the output from the first algorithm were the only data source for detecting objects of the first object type, the inconsistent false positive results related to the tree 108 could result in multiple problems, as previously discussed. Regions of the scene corresponding to the tree leaves (i.e., the image data 114 depicting such regions in the third image) would have uncertain object type status in the final image 100c (as further described below in connection with FIGS. 5-6) because they are interpreted as human faces in the final image 100c but not human faces in the other images 100a, 100b. However, as disclosed herein, using a background model to make the final decision about such regions reduces the problem of inconsistent false positives, and in this case, the false positives may be removed entirely from the third image frame 100c.

[0053] FIG. 3 illustrates such a background model 300. In the example of FIG. 3, the background model 300 defines, for each of a plurality of regions in a scene, whether the image data corresponding to the region in the last image frame 100c belongs to the background or the foreground. In the scene of FIG. 1, only the dog 104 and the car 106 move between the image frames 100a-c. The background model in FIG. 3 generally defines regions 304, 306 corresponding to the positions of the car 106 and the dog 104 in the last image 100c in FIG. 1 as the foreground, and the rest as the background 302, for ease of explanation only. The background model may be determined by applying a second algorithm (different from the first algorithm) to at least some of the plurality of images. Therefore, the background model may depend on the characteristics of the second algorithm. The complexity of the second algorithm may vary from a simple frame difference algorithm to a Gaussian mixture model that takes color and depth information into account. If the background model is determined using a different scaling of the scene compared to the image frames 100a-c, scaling parameters may be used to determine corresponding regions between the image frames 100a-c and the background model 300.

[0054] When the background model 300, as disclosed herein, is applied to image data 114 having uncertain object type status, the incorrect classification of the image data provided by the first algorithm may be canceled, and instead, the image data may be correctly classified as not being part of an object of the first object type (i.e., not being part of a human face) because the image data 114 is considered to be part of the background (indicated by the dashed region 308 in FIG. 3 , which spatially corresponds to the image data 114 shown in FIG. 1 ) and is therefore determined to not be part of an object of the first object type.

[0055] For image data in the last image 100c of Figure 3 that is not identified as having uncertain object type status, such as image data 116 in the last image 100c or image data depicting dog 104, the output from the first algorithm is relied upon. As a result, image data 116 is determined to be part of an object of the first object type (i.e., part of a human face), and the image data depicting the dog is determined to not be part of an object of the first object type.

[0056] FIG. 2 illustrates another embodiment of the techniques described herein, similar in many respects to the embodiment described in connection with FIG. 1. The second embodiment differs from the first embodiment in that the assessment made by the first algorithm relates to objects detected and tracked within the video stream, rather than specific regions of the scene. In other words, the first algorithm focuses on analyzing and classifying individual objects within the video, rather than regions or sections of the scene. FIG. 2 illustrates an example of a video sequence in which an object classification algorithm outputs varying classifications for several objects. FIG. 2 illustrates three images 200a-c from a video sequence capturing a scene. The scene includes five objects: a tree 222, a dog 218, a car 220, and two people 202 and 204. The objects are tracked in multiple image frames 100a-c. The images 200a-c are further enhanced with highlighting features 206, 208, 210, 212, 214 that indicate objects that the first algorithm has identified as being of the first object type in each of the image frames 100a-c. In the example of Figure 2, the first algorithm is configured to detect people.

[0057] If the output from the first algorithm were the only data source for detecting objects of the first object type, the results of inconsistent false positives related to the tree 222 and inconsistent false negatives related to the two people 202, 204 could result in multiple problems, as previously discussed. These three objects 202, 204, 220 have uncertain object type status in the final image 100c (as further described below in connection with FIGS. 5-6) because the object types of these objects are assessed differently throughout the video stream. However, as disclosed herein, using a background model to make a final determination about such objects reduces the problem of inconsistent false positives and false negatives, and in this case, the false positives and false negatives may be removed altogether from the third image frame 200c.

[0058] FIG. 4 shows a background model 400 that is similar to the background model 300 of FIG. 3. In the example of FIG. 4, the background model 400 defines, for each of a plurality of regions in a scene, whether the image data corresponding to the region in the last image frame 200c belongs to the background or the foreground. In the scene of FIG. 2, a dog 218, a car 220, and two people 202 and 204 move between image frames 100a-c. The background model in FIG. 4 generally defines regions 406, 408, 404, and 410 that correspond to the positions of the two people 202 and 204, the dog 218, and the car 220 in the last image 100c of FIG. 1 as the foreground, and defines the rest as the background 402.

[0059] When background model 400, as disclosed herein, is applied to tree 222 with uncertain object type status, the incorrect classification of the object provided by the first algorithm may be canceled, and instead, the object may be correctly classified as not being an object of the first object type (i.e., not containing a human face) because the image data corresponding to tree 222 is considered to be part of the background in final image 100c (indicated by dashed region 412 in FIG. 4, which spatially corresponds to the location and extent of tree 222 shown in FIG. 2), and is therefore determined to not be part of an object of the first object type.

[0060] When a background model, as disclosed herein, is applied to the persons 202, 204, each having an uncertain object type status, the missed classification of the object 202 provided by the first algorithm for the final image 200c may be corrected, and instead the object may be correctly classified as being an object of the first object type (i.e., including a human face), because the image data corresponding to each of the persons 202, 204 is considered to be part of the foreground in the final image 100c (indicated by dashed regions 406, 408 in FIG. 4, which spatially correspond to the positions and extents of the persons 202, 204 shown in FIG. 2), and is therefore determined to be part of an object of the first object type.

[0061] Additionally, FIG. 2 includes a dashed line 216 representing an effective range or detection range within which the first algorithm can accurately classify objects based on their features or characteristics in the image. In some embodiments, the technique described above in connection with FIG. 2 (in which the assessment performed by the first algorithm is associated with objects detected in the video stream rather than a specific region of the scene, as described in connection with FIG. 1 ) is applied primarily to objects located beyond a threshold distance from the camera that captured the video sequence of the scene. The threshold distance may be set to an effective range that is the same as or similar to the effective range of the first algorithm. Because moving people 202, 204 are located beyond this threshold, i.e., beyond the dashed line 216 in FIG. 2 , the first algorithm may be less reliable in correctly classifying moving objects of the first object type. Thus, this embodiment may reduce false negatives as described above.

[0062] In some examples, two aspects of the object detection techniques described in connection with Figures 1 and 2 may be combined. For example, a region-based technique may be used for image content near the range of effectiveness of a first algorithm, and an object-based technique is used for objects detected beyond the range of effectiveness of the first algorithm.

[0063] 5-6 illustrate two exemplary embodiments of lists of indicators 502, 602 generated by a first algorithm when analyzing a particular region of a captured scene in a video sequence or when analyzing a particular object tracked in a video sequence. As described above, in a region-based approach, the first algorithm may be configured to store, for each region in each image, an indicator of the difference between the probability that the image data (depicting the region in the image currently being analyzed) is part of an object of a first object type and a probability threshold in a list of indicators associated with the region. In an object-based approach, the first algorithm may be configured to store, for each tracked object in each image, an indicator of the difference between the probability that the object is an object of the first object type and a probability threshold in a list of indicators associated with the object.

[0064] Regardless of which approach is used, each region or object may be associated with a list 502, 602 as shown in FIG. 5 or FIG. 6. Each cell in each list 502, 602 represents the output from a first algorithm when analyzing an image of a plurality of images. The lists 502, 602 in FIGS. 5-6 include X=8 cells, indicating that eight images have been analyzed. In some embodiments, the list of indicators associated with a region or object is a FIFO list having 5 to 15 values. In other embodiments, the list has 15 to 30 values. Any suitable number of values ​​may be used depending on the use case, for example, based on the frame rate of the camera that captured the video sequence. When a new image is analyzed, the oldest value in the list of indicators (the left-most value in these examples) is deleted and the new value is inserted at the end of the list. Thus, the list may always contain values ​​from the X most recently analyzed images in the video stream.

[0065] FIG. 5 shows a list 502 in which each indicator indicates how far the assessed probability of a region / object being a first object type is from a probability threshold. In FIG. 1, a positive number indicates that the assessed probability is higher than the threshold (meaning that the first algorithm assessed the region / object to be of the first object type). A negative number indicates that the assessed probability is lower than the threshold (meaning that the first algorithm assessed the region / object to be not of the first object type). 0 may be counted as a positive number. The probability range is between 0 and 1, and the probability threshold is 0.5, meaning that the cell value may range from −0.5 to +0.5. Other scales and thresholds may be applied.

[0066] The list of indicators 502 may be used to determine whether a region / object in the last image of the multiple images represented by the list of indicators has an uncertain object type status. In other words, for a particular image, the results from that particular image combined with the results from the previous X-1 images in the video stream are used to determine whether a region / object in the particular image has an uncertain object type status.

[0067] In one embodiment, the process of determining the uncertain object type status for a region / object in the last image frame involves first checking how certain the first algorithm is when assessing whether the region / object (depicted in the last image frame) is of the first object type. If the first algorithm is sufficiently certain, the result from the first algorithm is relied upon. In FIG. 5 , the value in cell 504 corresponding to the last image frame is +0.15, which means the assessed probability is 0.65. If this is counted as sufficiently certain (the absolute value of the difference between the probability that the image data is part of an object of the first object type and the probability threshold exceeds the threshold difference), it is determined that the object / region to which list 502 relates is indeed of the first object type. If there is not sufficient certainty, or if the embodiment that first checks the probability of the last image frame is not used, further analysis of list 502 is required.

[0068] Analyzing the list may include determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences. The distribution measure may include any suitable statistical method, such as the mean (0.13 in the example of FIG. 3), the median (0.2), or a percentage distribution between positive and negative values ​​(75 / 25). If the distribution measure indicates a mix of positive and negative differences that falls within the uncertainty range, the region / object in the final image is determined to have an uncertain object type status; otherwise, it is not determined to have an uncertain object type status. As mentioned above, because the list includes positive and negative numbers, the uncertainty range typically includes thresholds on both the negative and positive sides. Therefore, the first algorithm may be considered to provide sufficiently consistent assessments for both negative and positive assessments. 5, the uncertainty range is between negative and positive numbers when mean and median values ​​are used (e.g., a mean or median value between −0.2 and +0.2 would result in the region / object of the first image data being determined to have an uncertain object type status). In the case of a distribution (without considering the magnitude of the values), the uncertainty range may indicate, for example, a mix between a 20 / 80 distribution and an 80 / 20 distribution of indicators indicating positive and negative differences. Because the distribution of the list in FIG. 5 falls within this range (75 / 25), the result may be that the region / object associated with list 502 is considered to have an uncertain object type status.

[0069] In some embodiments, the list of indicators is a list of binary values, with positive differences indicated by the first value of the binary value and negative differences indicated by the second value of the binary value. This is shown in Figure 6, which corresponds to list 502 of Figure 5, but in binary format instead. The same distribution measures, such as the mean (0.6) and the percentage distribution between positive and negative values ​​(75 / 25), may apply, and the uncertainty ranges may need to be adjusted according to the binary format compared to the format shown in Figure 5.

[0070] 7-8 show flowcharts of two methods for detecting an object of a first object type in a first image frame of a plurality of image frames according to the first and second aspects described herein, with Fig. 7 corresponding to the region-based aspect and Fig. 8 corresponding to the object aspect.

[0071] 7 illustrates a method 700 for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene. The method includes analyzing each region of a scene in each image frame in the video sequence with a first algorithm configured to detect the first object type. Stated differently, the method 700 may include, for each image frame of the plurality of image frames, analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates, for each region of the plurality of regions in the scene, a probability that image data corresponding to the region is part of an object of the first object type in the image frame, and the first algorithm identifies an object of the first object type in the image frame having a probability threshold for determining whether the image data is part of an object of the first object type. An indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold is then stored in a list of indicators associated with the region.

[0072] The method further includes providing a background model of the video sequence using a second algorithm when detecting an object of the first object type in a last image frame of the plurality of image frames S704. The background model may define, for each region of the plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground.

[0073] The method further includes, for each region of the scene, identifying S706 whether the image data corresponding to the region has an uncertain object type status based on the analysis by the first algorithm. The uncertain object type status may be based on an associated list of indicators.

[0074] The method further includes checking a background model to determine an object type status of the region of the first image if the region (i.e., the image data corresponding to the region) has an uncertain object type status S708. The background model may be used to determine that the image data is not part of an object of the first object type if the background model defines the image data as belonging to a background in the first image, and to determine that the image data is part of an object of the first object type if the background model defines the image data as belonging to a foreground in the first image.

[0075] In some examples, if a region (i.e., the image data corresponding to the region) does not have an uncertain object type status, the analysis of the region of the first image performed by the first algorithm is relied upon S710. This may mean determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type, and determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type.

[0076] In some examples, the method includes masking or highlighting S712 image data in the first image that is determined to be part of an object of the first object type.

[0077] FIG. 8 illustrates a method 800 for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames.

[0078] The method 800 includes analyzing S802 a tracked object in a scene in each frame in the video sequence with a first algorithm configured to detect a first object type. Stated differently, the method 800 may include, for each image frame of a plurality of image frames, analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates, for each object of the plurality of objects tracked in the image frame, a probability that the object is an object of the first object type, and the first algorithm identifies an object of the first object type in the image frame having a probability threshold for determining whether the object is an object of the first object type. An indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold is then stored in a list of indicators associated with the object.

[0079] The method further includes providing a background model of the video sequence using a second algorithm when detecting an object of the first object type in a last image frame of the plurality of image frames S804. The background model may define, for each region of the plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground.

[0080] The method further includes, for each tracked object in the scene, identifying S806 whether the tracked object has an uncertain object type status based on analysis of the first algorithm. The uncertain object type status may be based on an associated list of indicators.

[0081] The method further includes checking a background model to determine an object type status of the tracked object in the first image if the object has an uncertain object type status S808. The background model may be used to determine that the object is not an object of the first object type if the background model defines that image data corresponding to the object in the last image of the video sequence belongs to the background in the first image, and to determine that the object is an object of the first object type if the background model defines the image data as belonging to the foreground in the first image.

[0082] In some examples, if the object does not have an uncertain object type status, the analysis of the object in the first image performed by the first algorithm is relied upon S810. This may mean determining that the object is the first object type when the first algorithm determines that the object is the first object type, and determining that the object is not the first object type when the first algorithm determines that the object is not the first object type.

[0083] In some examples, the method includes masking or highlighting S812 the object in the first image that is determined to be of the first object type.

[0084] The methods illustrated in Figures 7-8, as well as any other methods or functionality described herein, may be stored as instructions on a non-transitory computer-readable storage medium such that the methods are performed when the instructions are executed on a device or system having processing capabilities. Such a device or system may comprise one or more processors. Accordingly, the features and methods described herein may be advantageously implemented in one or more computer programs executable on a programmable system that may include at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to a data storage system, at least one input device, such as a camera for capturing image frames / video sequences, and at least one output device, such as a display for displaying the potentially masked or highlighted image described herein. Suitable processors for executing a program of instructions include, by way of example, both general-purpose and special-purpose microprocessors, and the sole processor or one or more processors or cores of any type of computer. The processor may be supplemented by or incorporated in an ASIC (application-specific integrated circuit).

[0085] The above-described embodiments should be understood as illustrative examples of the present invention. Further embodiments of the present invention are contemplated. For example, the techniques described herein may be employed in any suitable object classification system, such as those used in autonomous vehicles, sports analysis, surveillance and security, and weather forecasting. It should be understood that any feature described in connection with any one embodiment may be used alone or in combination with other described features, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may be employed without departing from the scope of the present invention, as defined in the appended claims.

[0086] Enumerated List of Embodiments A. A method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the method comprising: For each image frame of the plurality of image frames, analyzing the image frames using a first algorithm to identify objects of a first object type in the image frames, wherein the first algorithm calculates, for each region of a plurality of regions in the scene, a probability that image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether the image data is part of an object of the first object type, and storing an indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list of indicators associated with the region; detecting an object of a first object type in a first image frame, the first image frame being a last image frame of a plurality of image frames; and detecting includes: providing a background model for the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data corresponding to the region in a first image frame belongs to the background or the foreground, the background model being defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames; for each region of the scene, identifying whether image data corresponding to the region has an uncertain object type status in the first image based on an associated list of indicators; Upon identifying that the image data has an uncertain object type status, determining that the image data is not part of an object of the first object type when the background model defines the image data as belonging to a background in the first image; determining that the image data is part of an object of the first object type when the background model defines the image data as belonging to a foreground in the first image; A method comprising:

[0087] B. Identifying the image data as not having uncertain object type status determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type; determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type; The method of embodiment A, further comprising:

[0088] C. Identifying whether the image data has an uncertain object type status in the first image comprises: determining whether an absolute value of a difference between the probability that the image data is part of an object of the first object type and a probability threshold exceeds a threshold difference based on the associated list of indicators; determining that the image data does not have uncertain object type status if the absolute value of the difference exceeds a threshold difference; if the difference does not exceed a threshold distance, determining a distribution measure between the indicators in the associated list of indicators indicating a positive difference and the indicators in the associated list of indicators indicating a negative difference; If the distribution measurement indicates a mixture of positive and negative differences that are included within the uncertainty range, then determine that the image data has an uncertain object type status; otherwise, determine that the image data does not have an uncertain object type status. The method of any one of embodiments A to B, comprising:

[0089] D. Identifying whether the image data has an uncertain object type status in the first image comprises: determining a distribution measure between indicators in the associated list of indicators showing a positive difference and indicators in the associated list of indicators showing a negative difference; If the distribution measurement indicates a mixture of positive and negative differences that are included within the uncertainty range, then determine that the image data has an uncertain object type status; otherwise, determine that the image data does not have an uncertain object type status. The method of any one of embodiments A to B, comprising:

[0090] E. The method of embodiment D, wherein the list of indicators is a list of binary values, a positive difference being indicated by a first value of the binary values ​​and a negative difference being indicated by a second value of the binary values.

[0091] F. The method of any one of embodiments C to E, wherein the uncertainty range exhibits a mix between a 20 / 80 distribution and an 80 / 20 distribution of indicators indicating positive and negative differences.

[0092] G. The method of any one of embodiments A to E, wherein the first algorithm includes a feature extraction process for extracting features of objects in the image and representing them as vectors of numbers.

[0093] H. The method of any one of embodiments A to G, wherein the second algorithm is a motion-based background detection algorithm.

[0094] I. The method of any one of embodiments A to H, wherein the list of indicators associated with the region is a FIFO list having 5 to 15 values.

[0095] J. The method of any one of embodiments A to I, further comprising masking or highlighting image data in the first image that is determined to be part of an object of the first object type.

[0096] K. The method of any one of embodiments A to J, implemented in a camera and wherein the video sequence is part of a live video stream captured by the camera.

[0097] L. A method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames, the method comprising: For each image frame of the plurality of image frames, analyzing the image frames using a first algorithm to identify objects of a first object type in the image frames, wherein the first algorithm calculates, for each object of a plurality of objects tracked in the image frames, a probability that the object is an object of the first object type, the first algorithm has a probability threshold for determining whether the object is an object of the first object type, and stores an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list of indicators associated with the object; detecting an object of a first object type in a first image frame, the first image frame being a last image frame of a plurality of image frames; and detecting includes: providing a background model for the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data corresponding to the region in a first image frame belongs to the background or the foreground, the background model being defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames; For each object of the plurality of objects tracked in the first image, identifying whether the object has an uncertain object type status based on a list of indicators associated with the object; Upon identifying an object as having an uncertain object type status, determining image data corresponding to the object in the first image; determining that the object is not an object of the first object type if the background model defines the image data as belonging to the background in the first image; determining that the object is an object of the first object type if the background model defines the image data as belonging to the foreground in the first image; A method comprising:

[0098] M. The method of embodiment L, wherein each of the tracked objects is located more than a threshold distance from the camera that captured the video sequence of the scene.

[0099] N. Identifying an object as not having uncertain object type status determining that the object is an object of the first object type when the first algorithm determines that the object is an object of the first object type; determining that the object is not an object of the first object type when the first algorithm does not determine that the object is an object of the first object type; The method of any one of embodiments L to M, further comprising:

[0100] O. Identifying whether an object has an uncertain object type status comprises: determining whether an absolute value of a difference between a probability that the object is an object of the first object type in the first image and a probability threshold exceeds a threshold difference based on the associated list of indicators; determining that the object does not have uncertain object type status if the absolute value of the difference exceeds a threshold difference; if the difference does not exceed a threshold distance, determining a distribution measure between the indicators in the associated list of indicators indicating a positive difference and the indicators in the associated list of indicators indicating a negative difference; If the distribution measurement shows a mixture of positive and negative differences that fall within the uncertainty range, then determine that the object has an uncertain object type status; otherwise, determine that the object does not have an uncertain object type status. The method of any one of embodiments L to N, comprising:

[0101] P. Identifying whether an object has an uncertain object type status comprises: determining a distribution measure between indicators in the associated list of indicators showing a positive difference and indicators in the associated list of indicators showing a negative difference; If the distribution measurement shows a mixture of positive and negative differences that fall within the uncertainty range, then determine that the object has an uncertain object type status; otherwise, determine that the object does not have an uncertain object type status. The method of any one of embodiments L to N, comprising:

[0102] Q. The method of embodiment P, in which the list of indicators is a list of binary values, a positive difference being indicated by a first value of the binary values ​​and a negative difference being indicated by a second value of the binary values.

[0103] R. The method of any one of embodiments O to Q, wherein the uncertainty range exhibits a mix between a 20 / 80 distribution and an 80 / 20 distribution of indicators indicating positive and negative differences.

[0104] S. The method of any one of embodiments L to R, wherein the first algorithm includes a feature extraction process for extracting features of objects in the image and representing them as vectors of numbers.

[0105] T. The method of any one of embodiments L to S, wherein the second algorithm is a motion-based background detection algorithm.

[0106] U. The method of any one of embodiments L to T, wherein the list of indicators associated with the object is a FIFO list having 5 to 15 values.

[0107] V. The method of any one of embodiments L-U, further comprising masking or highlighting objects in the first image determined to be of the first object type.

[0108] X. The method of any one of embodiments L to V, implemented in a camera and wherein the video sequence is part of a live video stream captured by the camera.

[0109] Y. A non-transitory computer-readable storage medium storing instructions for performing the method of any one of claims A through K or the method of any one of claims L through X when executed on a device having processing capability.

[0110] Z. A system for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the system comprising: one or more processors; one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the system to perform the method of any one of claims A through K or the method of any one of claims L through X; A system comprising:

Claims

1. A method (700) for detecting an object (102) of a first object type in a first image frame (100c) of a plurality of image frames (100a-c) in a video sequence capturing a scene, the method comprising: For each image frame of the plurality of image frames, analyzing the image frames using a first algorithm to identify objects of the first object type in the image frames, the first algorithm calculating, for each region of a plurality of regions in the scene, a probability that image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether image data is part of the object of the first object type, and storing an indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list of indicators associated with the region (302, 402); detecting an object of the first object type in the first image frame, the first image frame being a last image frame of the plurality of image frames; wherein said detecting comprises: providing a background model (300) of the video sequence, the background model defining, for each region of the plurality of regions in the scene, whether the image data in the first image frame corresponding to the region belongs to the background (302) or the foreground (304, 306), the background model being defined by applying a second algorithm to at least some of the plurality of image frames, the second algorithm being different from the first algorithm; and providing a background model (S704) of the video sequence. for each region of the scene, identifying whether the image data corresponding to the region in the first image frame has an uncertain object type status based on a plurality of indicators in the list of associated indicators, the list including a separate indicator for each image frame of the plurality of image frames (S706); Upon identifying that the image data has an uncertain object type status, determining that the image data is not part of an object of the first object type when the background model defines the image data as belonging to the background in the first image frame; determining (S708) that the image data is part of an object of the first object type when the background model defines the image data as belonging to the foreground in the first image frame; A method comprising:

2. Identifying image data as not having uncertain object type status (S701); determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of the object of the first object type; determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of the object of the first object type; The method of claim 1 further comprising:

3. identifying whether the image data has an uncertain object type status in the first image frame; determining whether an absolute value of the difference between the probability that the image data is part of an object of the first object type and the probability threshold exceeds a threshold difference, using an indicator from the list of associated indicators that indicates a difference between the probability that the image data is part of an object of the first object type and the probability threshold; determining that the image data does not have uncertain object type status if the absolute value of the difference exceeds the threshold difference; if the difference does not exceed the threshold difference, applying a statistical method to determine a distribution measure between indicators in the list of associated indicators that indicate a positive difference and indicators in the list of associated indicators that indicate a negative difference; determining that the image data has an uncertain object type status if the distribution measure indicates a mixture of positive and negative differences that are within an uncertainty range; otherwise, determining that the image data does not have an uncertain object type status; 2. The method of claim 1, wherein the uncertainty range comprises an upper threshold and a lower threshold, the upper and lower thresholds defining a statistical measure according to the statistical method.

4. identifying whether the image data has an uncertain object type status in the first image frame; determining a distribution measure between the indicators in the list of associated indicators that indicate a positive difference and the indicators in the list of associated indicators that indicate a negative difference; determining that the image data has an uncertain object type status if the distribution measure indicates a mixture of positive and negative differences that are within an uncertainty range; otherwise, determining that the image data does not have an uncertain object type status; The method of claim 1 , comprising:

5. 5. The method of claim 4, wherein the list of indicators is a list of binary values, a positive difference being indicated by a first one of the binary values ​​and a negative difference being indicated by a second one of the binary values.

6. The method of claim 3 , wherein the uncertainty range indicates a mix between a 20 / 80 distribution and an 80 / 20 distribution of the indicators indicating positive and negative differences.

7. The method of claim 1 , wherein the first algorithm includes a feature extraction process for extracting features of objects in an image and representing them as vectors of numbers.

8. The method of claim 1 , wherein the second algorithm is a motion-based background detection algorithm.

9. 2. The method of claim 1, wherein the list of indicators associated with a region is a FIFO list having between 5 and 15 values.

10. The method of claim 1 , further comprising masking or highlighting (S712) image data in the first image frame that is determined to be part of an object of the first object type.

11. The method of claim 1 , wherein the method is implemented in a camera and the video sequence is part of a live video stream captured by the camera.

12. A method (800) for detecting an object (202, 204) of a first object type in a first image frame (200c) of a plurality of image frames (200a-c) in a video sequence capturing a scene, the scene including a plurality of objects (202, 204, 218, 220, 222) tracked in the plurality of image frames, the method comprising: For each image frame of the plurality of image frames, analyzing the image frames using a first algorithm to identify objects of the first object type in the image frames, wherein the first algorithm calculates, for each object of the plurality of objects tracked in the image frames, a probability that the object is an object of the first object type, the first algorithm having a probability threshold for determining whether the object is an object of the first object type, and storing an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list of indicators associated with the object (302, 402); detecting an object of the first object type in the first image frame, the first image frame being a last image frame of the plurality of image frames; wherein said detecting comprises: providing a background model (400) of the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data in the first image frame corresponding to the region belongs to the background (400) or the foreground (404, 406, 408, 410), the background model being defined by applying a second algorithm to at least some of the plurality of image frames, the second algorithm being different from the first algorithm; and providing a background model (400) of the video sequence (S804), the background model defining, for each region of a plurality of regions in the scene, whether image data in the first image frame corresponding to the region belongs to the background (400) or the foreground (404, 406, 408, 410), the background model being defined by applying a second algorithm to at least some of the plurality of image frames, the second algorithm being different from the first algorithm. For each object of the plurality of objects tracked in the first image frames, identifying whether the object has an uncertain object type status based on the list of indicators associated with the object, the list including a separate indicator for each image frame of the plurality of image frames (S806); Identifying the object as having an uncertain object type status (S808); determining image data corresponding to the object in the first image frame; determining that the object is not an object of the first object type when the background model defines the image data as belonging to the background; determining that the object is an object of the first object type when the background model defines the image data as belonging to the foreground; A method comprising:

13. The method of claim 12 , wherein each of the tracked objects is located more than a threshold distance (216) from a camera that captured the video sequence of the scene.

14. A non-transitory computer-readable storage medium having stored thereon instructions for implementing the method of claim 1 or the method of claim 12 when executed on a device having processing capabilities.

15. 1. A system for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the system comprising: one or more processors; one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to perform the method of claim 1 or the method of claim 12; and A system comprising:

Images