Method for detecting object of first object type in video sequence, system and non-temporal computer readable storage medium

Abstract

To detect an object of a first object type in a video sequence.SOLUTION: A method analyzes each area of a scene in each image frame in a video sequence by a first algorithm for detecting a first object type, detects an object of the first object type in the last image frame among a plurality of image frames, provides a background model of the video sequence by using a second algorithm, identifies whether image data corresponding to the area has an uncertain object type status on the basis of an analysis by the first algorithm about each area of the scene, checks the background model in order to determine an object type status of an area of a first image in the case that the area has the uncertain object type status, depends on the analysis of the area of the first image to be performed by the first algorithm in the case of having no uncertain object type status, and masks or highlights image data determined to be a portion of the object of the first object type in the first image.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to object detection, and in particular to a method, system and non-transitory computer-readable storage medium for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene. [Background technology]

[0002] Object detection in video surveillance systems has evolved over the years as an essential component for a variety of applications, such as security, traffic monitoring, retail analytics, and smart cities. The main objective of these systems is to identify and track objects of interest, such as people, vehicles, or animals, in a video stream.

[0003] Object classification is often an essential component of an object detection system. Object detection involves identifying objects of interest in an image or video frame and assigning them to a particular category or class. The combination of object localization (finding the location of an object) and object classification (identifying the class of an object) enables a complete object detection system.

[0004] One problem with object detection systems is incorrectly classified objects, i.e., false positives or false negatives. Such classifications can, for example, negatively impact user experience. In systems where a human operator reviews or interacts with the video output, incorrectly classified objects can result in confusion, fatigue, and frustration. This can reduce user confidence in the system and can cause the operator to miss genuine objects of interest due to an increased number of false alarms.

[0005] Therefore, improvements in this context are needed. Summary of the Invention

[0006] In view of the above, it would be advantageous to overcome or at least reduce one or more of the above-described disadvantages as set forth in the accompanying independent patent claims.

[0007] According to a first aspect of the present invention there is provided a method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the method comprising: for each image frame of the plurality of image frames, analysing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates for each region of a plurality of regions in the scene a probability that image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether the image data is part of an object of the first object type, and storing an indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list of indicators associated with the region.

[0008] The method further includes detecting an object of the first object type in the first image frame, where the first image frame is a last image frame of the plurality of image frames, where detecting includes providing a background model of the video sequence, where the background model defines for each region of a plurality of regions in the scene whether image data corresponding to the region in the first image frame belongs to the background or the foreground, the background model being defined by applying a second algorithm to at least some of the plurality of image frames, the second algorithm being different from the first algorithm; and for each region of the scene, identifying whether image data corresponding to the region in the first image has an uncertain object type status based on an associated list of indicators; and upon identifying that the image data has an uncertain object type status, determining that the image data is not part of an object of the first object type when the background model defines the image data as belonging to the background in the first image; and determining that the image data is part of an object of the first object type when the background model defines the image data as belonging to the foreground in the first image.

[0009] In object classification applications designed to detect specific object types, problems can arise when the class of an object alternates between frames. Some objects may have an appearance that is similar to the target object type, causing the classification algorithm to generate varying probabilities. These probabilities may exceed a threshold to classify the object as the target type in one frame, and then fall below the threshold to classify the object as a different type in the next frame. Factors contributing to these changes may include the viewing angle of the object, image sensor noise, small changes in segmentation mask determination, or variations in the appearance of the object.

[0010] Such inconsistent classification can lead to confusion for operators monitoring the output and can adversely affect system performance, decision-making, and privacy, among other problems.

[0011] Advantageously, using two different algorithms to determine object type may increase the robustness of the method. The second algorithm, which provides a background model of the video sequence, may be considered as a backup in case the first algorithm outputs different assessments (probabilities above and below a probability threshold) across multiple image frames for the corresponding image content. Such varying outputs between multiple image frames for a particular region of the scene may lead to the region of the scene depicted in the currently processed image frame (referred to as the first image frame, which is the latest image frame in the video sequence) getting an "uncertain object type status". By using a previous frame in the video sequence to determine whether the region of the scene depicted in the first frame (i.e., the image data depicting the region in the first frame) should get an uncertain object type status, prediction of future states is avoided, thus resulting in a low complexity and robust method.

[0012] It is important to clarify that the term "first image frame" as used herein does not refer to the order of a frame in a video sequence. Rather, it is merely a means of distinguishing one frame from another for purposes of describing or claiming the invention. The order of frames in a sequence is determined by their temporal relationship to one another, with the first frame being captured or displayed before the second frame, and so on until the last frame is reached.

[0013] By the term "uncertain object type status", in the context of this specification, it should be understood that the output from the first algorithm results in a mixture of classification results during analysis of multiple image frames, for example where an area of ​​the scene depicted in some of the image frames is classified as not being part of an object of the first object type, while in other of the image frames the area is classified as being part of an object of the first object type.

[0014] The degree of mixing that results in an uncertain object type status, e.g., the ratio between positive and negative classifications, the magnitude of the difference between the probability that the image data is part of an object of the first object type and a probability threshold, etc., depends on the use case and application requirements.

[0015] When an uncertain object type status is determined for particular image data of a first image, the background model is used to determine whether an object of the first object type has been detected in the image data.

[0016] Advantageously, in the method, false positive identification of a static object that is similar in appearance to an object of the first object type can be avoided because the background model can determine that the image data depicting the static object in the first image frame belongs to the background.Furthermore, the background model can prevent false negative identification of moving objects belonging to the first object type even when their appearance in one or more of the image frames does not match the assessment criteria of the first algorithm.This is because the background model can determine that the image data representing the moving object in the first image frame belongs to the foreground, thus avoiding false negative identification.

[0017] Furthermore, the list of indicators may be continually updated as new image frames are added to the video sequence, potentially using a first-in, first-out (FIFO) approach, such that the method can effectively adapt to new objects or other scene changes captured within the video sequence.

[0018] In some embodiments, upon identifying the image data as not having an uncertain object type status, the method further includes determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type, and determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type.

[0019] In other words, if the image data in the first image is not identified as having an uncertain object type status, then the output from the first algorithm for the image data is trusted and the image data is determined to be part of an object of the first object type if the first algorithm made this assessment. Advantageously, a consistent assessment from the first algorithm can be relied upon and false positives and false negatives can be avoided.

[0020] In some examples, the step of identifying whether the image data has an uncertain object type status in the first image includes determining whether an absolute value of a difference between the probability that the image data is part of an object of the first object type and a probability threshold exceeds a threshold difference based on the associated list of indicators, and determining that the image data does not have an uncertain object type status if the difference exceeds the threshold difference, determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences if the absolute value of the difference does not exceed the threshold distance, and determining that the image data has an uncertain object type status if the distribution measure indicates a mix of positive and negative differences that are within the uncertainty range, and otherwise determining that the image data does not have an uncertain object type status.

[0021] Advantageously, if the first algorithm is sufficiently certain in its assessment of the image data of the first image (the difference between the probability that the image data in the first image is part of an object of the first object type and the probability threshold exceeds the threshold difference), this assessment is relied upon and the image data is not considered to have an uncertain object type status. This embodiment may reduce the complexity of the method, since if the first algorithm assesses a sufficiently high or sufficiently low probability that the image data of the first image is part of an object of the first object type, then an examination of the remaining list of indicators (i.e. the assessment made for the previous frame) and a check of the background model are not required. Furthermore, false positives and false negatives may be avoided.

[0022] The threshold difference may depend on the requirements of the application in which the method is implemented and / or the reliability metric of the first algorithm. The threshold difference may be set, for example, to 0.3 on a scale of 0 to 1. If the probability threshold is 0.5, this means that a probability of 0.8 or greater, or 0.2 or less, is considered "high enough" or "low enough", respectively. Other threshold differences are equally possible and depend on the use case.

[0023] If the first algorithm is not sufficiently certain, the list of indicators is analyzed to determine whether the image data should be considered to have an uncertain object type status. The distribution between positive and negative differences is analyzed using any suitable statistical method, for example, determining the ratio between negative and positive values, the mean of the list, the median of the list, the sum of the values ​​in the list, etc. If the distribution measure falls within the uncertainty range, the image data is considered to have an uncertain object type status, otherwise the image data is not considered to have an uncertain object type status.

[0024] Since the list indicates positive and negative differences and the first algorithm may be to classify the image data in two classes (first object type or non-first object type), the uncertainty range usually includes a low threshold and a high threshold. For example, if a ratio is determined, the uncertainty range may indicate a mixture between a 20 / 80 distribution and an 80 / 20 distribution of indicators indicating positive and negative differences. This means that the image data is considered uncertain if the ratio is between 20 / 80 and 80 / 20 (20 / 80, 30 / 70, 50 / 50, 60 / 40, 80 / 20, etc., but not 100 / 0, 0 / 100, 10 / 90 or 95 / 5). Any other suitable ratio for the range threshold may be used, such as 10 / 90, 25 / 75, etc. If the average value is used, the uncertainty range may include a lower value of -0.2 and a higher value of 0.3, so that an average value between -0.2 and +0.3 would be considered uncertain, but not values ​​outside that range (-0.3, +0.35, etc.).

[0025] In some embodiments, the step of identifying whether the image data has uncertain object type status in the first image includes determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences, and determining that the image data has uncertain object type status if the distribution measure indicates a mix of positive and negative differences that fall within the uncertainty range, and otherwise determining that the image data does not have uncertain object type status.

[0026] In this embodiment, the list of indicators is constantly analyzed, which may provide a more reliable assessment of the image data. Furthermore, this embodiment allows the list of indicators to be a list of binary values, with positive differences being indicated by a first value of the binary value and negative differences being indicated by a second value of the binary value. This may reduce the complexity of the method for both storing the list of indicators as well as determining the distribution measure of the list.

[0027] In some embodiments, the first algorithm includes a feature extraction process to extract features of objects in the image and represent them as vectors of numbers. This means that the first algorithm may include an artificial intelligence (AI) or machine learning (ML) algorithm trained to detect objects of the first object type in the image. AI / ML is a suitable technique for classifying objects in images and can be relatively easily trained on large datasets of images labeled with objects of interest. Suitable AI / ML algorithms include Haar cascade, Histogram of Oriented Gradients (HOG), Local Binary Pattern Histogram (LBPH), Convolutional Neural Networks (CNN) and Transformer type models (taking sequential data such as video streams as input).

[0028] In some embodiments, the second algorithm is a motion-based background detection algorithm. Examples of such algorithms include using a time-average filter, a frame differencing method, a mean filter, a running Gaussian mean, and a background mixture model (such as a Gaussian mixture model (GMM)). Other possible algorithms include an adaptive multi-band binary (AMBB) algorithm and an optical flow method. Using a motion-based background detection algorithm may provide a good backup algorithm if the first algorithm outputs uncertain results as described above.

[0029] In some embodiments, the list of indicators associated with the regions is a FIFO list having between 5 and 15 values. In other embodiments, the FIFO list may have more values, such as 20, 30, etc. Using a FIFO data structure may provide a simple, efficient, and effective method for calculating the distribution measures described herein, especially in real-time applications where new data points are being added continuously.

[0030] In some embodiments, the method further includes masking or highlighting in the first image image data determined to be part of an object of the first object type. When dealing with these types of applications, it can be frustrating for an operator if a particular object is masked or highlighted in one frame but not in the next. This inconsistency can be annoying and draw unnecessary attention. Furthermore, in masking applications, there are potential privacy concerns if an object is only occasionally masked. The method may solve or at least reduce one or some of these shortcomings.

[0031] In some embodiments, the method is implemented in a camera and the video sequence is part of a live video stream captured by the camera.

[0032] According to a second aspect of the present invention, there is provided a method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames, the method comprising: for each image frame of the plurality of image frames, analysing the image frame using a first algorithm to identify an object of the first object type in the image frame, the first algorithm calculating, for each object of the plurality of objects tracked in the image frame, a probability that the object is an object of the first object type, the first algorithm having a probability threshold for determining whether the object is an object of the first object type, and storing an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list of indicators associated with the object.

[0033] The method further includes detecting an object of the first object type in a first image frame, where the first image frame is a last image frame of the plurality of image frames, and the detecting includes providing a background model of the video sequence, where the background model defines, for each region of a plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground, and the background model is defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames.

[0034] The method further includes, for each object of the plurality of objects tracked in the first image, identifying whether the object has an uncertain object type status based on the list of indicators associated with the object, and upon identifying that the object has an uncertain object type status, determining image data corresponding to the object in the first image, determining that the object is not an object of the first object type if the background model defines the image data as belonging to a background in the first image, and determining that the object is an object of the first object type if the background model defines the image data as belonging to a foreground in the first image.

[0035] The method of the second aspect is similar to the method of the first concept, and the same or corresponding advantages can be achieved. The difference is that in the second aspect, the assessment from the first algorithm relates to the object detected in the video stream, not to the area of ​​the scene. Advantageously, the method of the second aspect can handle both stationary and moving objects. As in the first aspect, a background model is used to assess whether a tracked object with an uncertain object type status should be considered as background or foreground in the first image frame. If the image data corresponding to the object should be considered as foreground, the object is determined to be the first object type, and if not, is not determined to be the first object type.

[0036] In some embodiments, each of the tracked objects is located beyond a threshold distance from the camera that captured the video sequence of the scene. For example, the first algorithm may have an effective range or detection range within which the first algorithm can accurately classify objects based on their features or characteristics in the image. This range is typically determined by the resolution and quality of the camera and the performance of the object classification algorithm. For objects outside the detection range, factors such as poor image quality, occlusion, and changing lighting conditions may reduce the accuracy of the classification. In an embodiment, the threshold distance may be the same as or close to the detection range of the first algorithm. Using this embodiment, objects that may be classified as the first object type and are not considered background may be determined to be the first object type. As a result, false negatives may be reduced. This may be particularly important in scenarios where objects of the first object type should be masked, for example for privacy reasons.

[0037] The second embodiment may generally have the same features and advantages as the first embodiment.

[0038] According to a third aspect of the present invention there is provided a non-transitory computer readable storage medium storing instructions for performing a method according to the first or second aspects when executed on a device having processing capability.

[0039] According to a fourth aspect of the present invention, there is provided a system for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the system comprising one or more processors and one or more non-transitory computer-readable media storing computer-executable instructions which, when executed by the one or more processors, cause the system to perform a method of the first or second aspect.

[0040] The third and fourth aspects may generally have the same features and advantages as the first aspect.

[0041] Other objects, features and advantages of the present invention will become apparent from the following detailed disclosure, the attached dependent claims as well as the drawings.

[0042] In general, all terms used in the claims should be interpreted according to their ordinary meaning in the art, unless expressly defined otherwise herein. All references to "a / an / the [element, device, component, means, step, etc.]" should be openly interpreted as referring to at least one instance of the element, device, component, means, step, etc., unless expressly stated otherwise.

[0043] The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

[0044] The above, as well as additional objects, features, and advantages of the present invention will be better understood from the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the accompanying drawings, in which like reference numerals are used for similar elements, and in which: [Brief description of the drawings]

[0045] [Figure 1] FIG. 2 illustrates a video sequence including a plurality of images in which objects of a first object type are identified by a first algorithm according to a first embodiment. [Diagram 2] FIG. 2 illustrates a video sequence including a plurality of images in which objects of a first object type are identified by a first algorithm according to a second embodiment. [Diagram 3] FIG. 2 shows a background model of the video sequence in FIG. [Figure 4] FIG. 3 shows a background model of the video sequence in FIG. 2. [Diagram 5] FIG. 1 illustrates a list of indicators, according to an embodiment. [Figure 6] FIG. 1 illustrates a list of indicators, according to an embodiment. [Figure 7] 4 is a flowchart of a method for detecting an object of a first object type in a first image frame of a plurality of image frames according to an embodiment. [Figure 8] 4 is a flowchart of a method for detecting an object of a first object type in a first image frame of a plurality of image frames according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] Object classification systems may output false positives or false negatives for a variety of reasons, including imbalanced training data, similar object appearances, occlusions, lighting differences, etc. For example, objects with similar visual features or appearances may confuse the classification system, resulting in false positives or false negatives. Models may struggle to distinguish between objects with similar shapes, textures, or colors, especially under certain lighting conditions or from certain viewpoints.

[0047] In many object classification systems, especially those where the output is monitored by an operator, consistent false positives or false negatives may be ignored by the operator and may not attract much attention. However, inconsistent false positives or false negatives may be more difficult to ignore and may lead to user frustration, reduced trust in the system, and reduced adoption of the technology. Furthermore, in real-time applications such as surveillance applications, inconsistent false positives or false negatives may result in unstable decision making as the system constantly re-evaluates the presence or absence of objects of a particular object type. This may lead to inefficient or potentially dangerous decisions.

[0048] FIG. 1 illustrates an example of a video sequence in which an object classification algorithm outputs a variation classification for an object. FIG. 1 illustrates three images 100a-c from a video sequence capturing a scene. The scene includes four objects, namely a tree 108, a dog 104, a car 106 driving on a road, and a person 102. The images 100a-c are further enhanced with highlighting features 110, 112, 114, 116 indicating image regions (image data) where the first algorithm has identified an object of the first object type in each of the image frames 100a-c. In the example of FIG. 1, the first algorithm is configured to detect human faces. This detection can serve various purposes, such as monitoring restricted areas where access is prohibited and alerting an operator when an individual enters there, which can also trigger an alert. Additionally, face detection can be employed for privacy purposes, allowing for masking or blurring of faces to protect the identity of individuals. It should be noted that the first algorithm may be configured to detect other object types, such as cars, animals, etc., depending on the use case of the object classification system.

[0049] The first algorithm may be configured to identify objects of the first object type in the image frames 100a-c by calculating, for each region of a plurality of regions in a scene captured by the image frames 100a-c, a probability that image data corresponding to the region is part of an object of the first object type in the image frames 100a-c. The first algorithm may have a probability threshold for determining whether the image data is part of an object of the first object type. The first algorithm may be a machine learning or artificial intelligence model that provides various technical advantages such as improved accuracy, adaptability to new data, ability to handle complex relationships and scalability. Typically, such an algorithm, or other suitable algorithm, classifies the object type of an object with a determined probability, for example 0.7 or 0.2 on a scale of 0 to 1. If the determined probability exceeds the probability threshold, for example 0.5, the object is determined to be of the corresponding object type, otherwise it is determined to not be of the corresponding object type. The probability threshold may be fixed for all images or may vary based on, for example, the number of objects in the image, the quality of the image, the lighting conditions of the image, etc.

[0050] The first algorithm may include a feature extraction process to extract features of objects in the image frames 100a-c and represent them as vectors of numbers, and then the vectors of numbers (feature vectors) may be used to determine the probability that the image data of the object is part of an object of the first object type in the image frames 100a-c. The object locations may be mapped to regions of the scene. If the camera that captured the images is a static camera (as in the example of FIG. 1), each region of the scene may be represented by the same pixel index or pixel index range in each image frame 100a-c. In the case of a moving camera, motion data may be used in addition to the pixel index to determine corresponding pixel regions between the image frames 100a-c. If the image frames 100a-c were captured at different resolutions, a scaling parameter may be further used to determine corresponding pixel regions between the image frames 100a-c.

[0051] However, no matter how well the first algorithm is trained, the model may struggle to distinguish between objects that have similar shapes, textures, or colors, especially under certain lighting conditions or from certain viewpoints. This is illustrated in Fig. 1, where image data 114 depicting part of the leaves of a tree 108 is interpreted as a human face in the last image 100c of the tree images 100a-c. Apart from this incorrect classification, the first algorithm correctly interprets other areas of the scene as depicted in the three images 100a-c, and classifies only the image data 110, 112, 116 corresponding to areas where the person 102 is present in the scene as being part of an object of the first object type.

[0052] If the output from the first algorithm were the only data source for detecting objects of the first object type, the inconsistent false positive results relating to the tree 108 may result in multiple problems, as previously described. Regions of the scene corresponding to the tree leaves (i.e., image data 114 depicting such regions in the third image) have an uncertain object type status in this image 100c (as further described below in connection with Figures 5-6) because they are interpreted as a human face in the final image 100c but not a human face in the other images 100a, 100b. However, as disclosed herein, using a background model to make the final decision about such regions may reduce the problem of inconsistent false positives, in which case the false positives may be removed from the third image frame 100c as a whole.

[0053] FIG. 3 illustrates such a background model 300. In the example of FIG. 3, the background model 300 defines, for each region of a plurality of regions in a scene, whether the image data corresponding to the region in the last image frame 100c belongs to the background or the foreground. In the scene of FIG. 1, only the dog 104 and the car 106 move between the image frames 100a-c. The background model in FIG. 3 generally defines the regions 304, 306 corresponding to the positions of the car 106 and the dog 104 in the last image 100c in FIG. 1 as the foreground and the rest as the background 302, but this is merely for ease of explanation. The background model may be determined by applying a second algorithm (different from the first algorithm) to at least some of the plurality of images. Thus, the background model may depend on the characteristics of the second algorithm. The complexity of the second algorithm may vary from a simple frame difference algorithm to a Gaussian mixture model taking into account color and depth information. If the background model is determined using a different scaling of the scene compared to the image frames 100a-c, the scaling parameters may be used to determine corresponding regions between the image frames 100a-c and the background model 300.

[0054] When the background model 300 is applied to image data 114 having uncertain object type status as disclosed herein, the incorrect classification of the image data provided by the first algorithm may be cancelled and the image data may instead be correctly classified as not being part of an object of the first object type (i.e., not being part of a human face) because the image data 114 is considered to be part of the background (indicated by the dashed region 308 in FIG. 3 , which spatially corresponds to the image data 114 shown in FIG. 1 ) and is therefore determined to not be part of an object of the first object type.

[0055] For image data in the last image 100c of Figure 3 that is not identified as having an uncertain object type status, such as image data 116 in the last image 100c of Figure 3 or image data depicting the dog 104, the output from the first algorithm is relied upon. As a result, the image data 116 is determined to be part of an object of the first object type (i.e., is part of a human face) and the image data depicting the dog is determined to not be part of an object of the first object type.

[0056] FIG. 2 illustrates another embodiment of the techniques described herein, similar in many respects to the embodiment described in connection with FIG. 1. The second embodiment differs from the first embodiment in that the assessment made by the first algorithm relates to objects detected and tracked in the video stream, rather than to specific regions of the scene. In other words, the first algorithm focuses on analyzing and classifying individual objects in the video, rather than regions or sections of the scene. FIG. 2 illustrates an example of a video sequence in which an object classification algorithm outputs a variation classification for several objects. FIG. 2 illustrates three images 200a-c from a video sequence capturing a scene. The scene includes five objects, namely a tree 222, a dog 218, a car 220, and two people 202, 204. The objects are tracked in multiple image frames 100a-c. The images 200a-c are further enhanced with highlighting features 206, 208, 210, 212, 214 that indicate objects that the first algorithm has identified as being of the first object type in each of the image frames 100a-c. In the example of Figure 2, the first algorithm is configured to detect people.

[0057] If the output from the first algorithm was the only data source for detecting objects of the first object type, the results of inconsistent false positives related to the tree 222 and inconsistent false negatives related to the two people 202, 204 may result in multiple problems, as previously described. These three objects 202, 204, 220 have an uncertain object type status in the final image 100c (as further described below in connection with Figures 5-6) because the object types of these objects are assessed differently throughout the video stream. However, as disclosed herein, using a background model to make a final decision about such objects may reduce the problem of inconsistent false positives and false negatives, in which case the false positives and false negatives may be removed from the third image frame 200c as a whole.

[0058] FIG. 4 shows a background model 400, which is similar to the background model 300 of FIG. 3. In the example of FIG. 4, the background model 400 defines, for each region of a plurality of regions in a scene, whether the image data corresponding to the region in the last image frame 200c belongs to the background or the foreground. In the scene of FIG. 2, a dog 218 and a car 220, as well as two people 202, 204, move between the image frames 100a-c. The background model in FIG. 4 generally defines the regions 406, 408, 404, 410, which correspond to the positions of the two people 202, 204, the dog 218 and the car 220 in the last image 100c in FIG. 1, as the foreground, and the rest as the background 402.

[0059] When the background model 400 is applied to the tree 222 having uncertain object type status as disclosed herein, the incorrect classification of the object provided by the first algorithm may be cancelled and the object may instead be correctly classified as not being an object of the first object type (i.e., not including a human face) because the image data corresponding to the tree 222 is deemed to be part of the background in the final image 100c (indicated by the dashed region 412 in FIG. 4, which spatially corresponds to the location and extent of the tree 222 shown in FIG. 2) and is therefore determined to not be part of an object of the first object type.

[0060] When a background model is applied to the persons 202, 204, each having an uncertain object type status, as disclosed herein, the missed classification of the object 202 provided by the first algorithm for the final image 200c may be corrected, and the object may instead be correctly classified as being an object of the first object type (i.e., containing a human face), because the image data corresponding to each of the persons 202, 204 is deemed to be part of the foreground in the final image 100c (indicated by the dashed regions 406, 408 in FIG. 4, which spatially correspond to the positions and extents of the persons 202, 204 shown in FIG. 2), and is therefore determined to be part of an object of the first object type.

[0061] Furthermore, FIG. 2 includes a dashed line 216 representing an effective range or detection range within which the first algorithm can correctly classify objects based on their features or characteristics in the image. In some embodiments, the technique described above in connection with FIG. 2 (where the assessment made by the first algorithm is associated with objects detected in the video stream, rather than with a particular region of the scene, as described in connection with FIG. 1) is applied primarily to objects located beyond a threshold distance from the camera that captured the video sequence of the scene. The threshold distance may be set to an effective range that is the same as or similar to the effective range of the first algorithm. Since moving people 202, 204 are located beyond this threshold, i.e., beyond the dashed line 216 in FIG. 2, the first algorithm may be less reliable in correctly classifying moving objects of the first object type. Thus, this embodiment may reduce false negatives as described above.

[0062] In some examples, the two aspects of the object detection techniques described in connection with Figures 1 and 2 may be combined. For example, the region-based technique may be used for image content on the near side of the range of effectiveness of the first algorithm, and the object-based technique is used for objects detected beyond the range of effectiveness of the first algorithm.

[0063] 5-6 show two exemplary embodiments of lists of indicators 502, 602 generated by the first algorithm when analyzing a particular region of a captured scene in a video sequence or when analyzing a particular object tracked in a video sequence. As mentioned above, the first algorithm may be configured to store, in a region-based approach, for each region in each image, an indicator of the difference between the probability that the image data (depicting the region in the image currently being analyzed) is part of an object of the first object type and a probability threshold in a list of indicators associated with the region. In an object-based approach, the first algorithm may be configured to store, in a list of indicators associated with the object, for each tracked object in each image, an indicator of the difference between the probability that the object is an object of the first object type and a probability threshold.

[0064] Regardless of which approach is used, each region or object may be associated with a list 502, 602 as shown in FIG. 5 or FIG. 6. Each cell in each list 502, 602 represents the output from a first algorithm when analyzing an image of the plurality of images. The lists 502, 602 in FIG. 5-6 include X=8 cells, indicating that 8 images have been analyzed. In some embodiments, the list of indicators associated with the region or object is a FIFO list having 5-15 values. In other embodiments, the list has 15-30 values. Any suitable number of values ​​may be used depending on the use case, for example based on the frame rate of the camera that captured the video sequence. When a new image is analyzed, the oldest value in the list of indicators (the left-most value in these examples) is deleted and the new value is inserted at the end of the list. Thus, the list may always contain values ​​from the X most recent analyzed images in the video stream.

[0065] FIG. 5 shows a list 502 in which each indicator indicates how far the assessment of the probability that the region / object is the first object type is from a probability threshold. In FIG. 1, a positive number indicates that the assessed probability is higher than the threshold (meaning that the first algorithm assessed the region / object to be the first object type). A negative number indicates that the assessed probability is lower than the threshold (meaning that the first algorithm assessed the region / object to not be the first object type). 0 may be counted as a positive number. The probability range is between 0 and 1, and the probability threshold is 0.5, which means that the value of the cell may range from -0.5 to +0.5. Other scales and thresholds may be applied.

[0066] The list of indicators 502 may be used to determine whether a region / object in the last image of the multiple images that the list of indicators represents has an uncertain object type status. In other words, for a particular image, the results from that particular image in combination with the results from the previous X-1 images in the video stream are used to determine whether a region / object in the particular image has an uncertain object type status.

[0067] In one embodiment, the process of determining the uncertain object type status for a region / object in the last image frame includes first checking how certain the first algorithm is when assessing whether the region / object (depicted in the last image frame) is the first object type. If the first algorithm is sufficiently certain, the result from the first algorithm is relied upon. In FIG. 5, the value of the cell 504 corresponding to the last image frame is +0.15. This means that the assessed probability is 0.65. If this is counted as sufficiently certain (the absolute value of the difference between the probability that the image data is part of an object of the first object type and the probability threshold exceeds the threshold difference), it is determined that the object / region to which the list 502 relates is indeed the first object type. If it is not sufficiently certain, or if the embodiment of first checking the probability of the last image frame is not used, further analysis of the list 502 is required.

[0068] The analysis of the list may include determining a distribution measure between indicators in the associated list of indicators indicating positive differences and indicators in the associated list of indicators indicating negative differences. The distribution measure may include any suitable statistical method, such as the mean (0.13 in the example of FIG. 3), the median (0.2), or a percentage distribution between positive and negative values ​​(75 / 25). If the distribution measure indicates a mix of positive and negative differences that falls within the uncertainty range, the region / object in the last image is determined to have an uncertain object type status, otherwise it is not determined to have an uncertain object type status. As mentioned above, since the list includes positive and negative numbers, the uncertainty range typically includes thresholds on both the negative and positive sides. Thus, the first algorithm may be considered to provide a sufficiently consistent assessment for both negative assessments as well as positive assessments. As a result, in the case of Figure 5, the uncertainty range is between negative and positive numbers if the mean and median are used (e.g., a mean or median value between -0.2 and +0.2 would result in the regions / objects of the first image data being determined to have an uncertain object type status). In the case of a distribution (without considering the magnitude of the values), the uncertainty range may indicate, for example, a mix between a 20 / 80 to 80 / 20 distribution of indicators indicating positive and negative differences. Since the distribution of the listing in Figure 5 is within this range (it is 75 / 25), the result may be that the regions / objects associated with listing 502 are considered to have an uncertain object type status.

[0069] In some embodiments, the list of indicators is a list of binary values, with positive differences indicated by a first value of the binary value and negative differences indicated by a second value of the binary value. This is shown in Figure 6, which corresponds to list 502 of Figure 5, but in binary format instead. The same distribution measures may apply, such as the mean (0.6) and the percentage distribution between positive and negative values ​​(75 / 25), and the uncertainty ranges may need to be adjusted according to the binary format compared to the format shown in Figure 5.

[0070] Figures 7-8 show flow charts of two methods for detecting an object of a first object type in a first image frame of a plurality of image frames according to the first and second aspects described herein, Figure 7 corresponds to a region-based aspect and Figure 8 corresponds to an object aspect.

[0071] 7 shows a method 700 for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene. The method includes analyzing each region of a scene in each image frame in the video sequence with a first algorithm configured to detect a first object type S702. Stated differently, the method 700 may include analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame for each image frame of the plurality of image frames, the first algorithm calculating for each region of the plurality of regions in the scene a probability that image data corresponding to the region is part of an object of the first object type in the image frame, and the first algorithm identifying an object of the first object type in the image frame having a probability threshold for determining whether the image data is part of an object of the first object type. An indicator of the difference between the probability that the image data is part of an object of the first object type and the probability threshold is then stored in a list of indicators associated with the region.

[0072] The method further includes providing a background model of the video sequence using a second algorithm when detecting an object of the first object type in a last image frame of the plurality of image frames S704. The background model may define, for each region of the plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground.

[0073] The method further includes, for each region of the scene, identifying S706 whether the image data corresponding to the region has an uncertain object type status based on the analysis by the first algorithm. The uncertain object type status may be based on an associated list of indicators.

[0074] The method further includes checking a background model to determine an object type status of the region of the first image if the region (i.e., the image data corresponding to the region) has an uncertain object type status S708. The background model may be used to determine that the image data is not part of an object of the first object type if the background model defines the image data as belonging to a background in the first image, and to determine that the image data is part of an object of the first object type if the background model defines the image data as belonging to a foreground in the first image.

[0075] In some examples, if a region (i.e., the image data corresponding to the region) does not have an uncertain object type status, the analysis of the region of the first image performed by the first algorithm is relied upon S710. This may mean determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type, and determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type.

[0076] In some examples, the method includes masking or highlighting S712 image data in the first image that is determined to be part of an object of the first object type.

[0077] FIG. 8 shows a method 800 for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames.

[0078] The method 800 includes analyzing tracked objects in a scene in each frame in the video sequence with a first algorithm configured to detect a first object type S802. Stated differently, the method 800 may include, for each image frame of a plurality of image frames, analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame, where the first algorithm calculates, for each object of the plurality of objects tracked in the image frame, a probability that the object is an object of the first object type, and the first algorithm identifies an object of the first object type in the image frame having a probability threshold for determining whether the object is an object of the first object type. An indicator of the difference between the probability that the image data is part of an object of the first object type and the probability threshold is then stored in a list of indicators associated with the object.

[0079] The method further includes providing a background model of the video sequence using a second algorithm when detecting an object of the first object type in a last image frame of the plurality of image frames S804. The background model may define, for each region of the plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background or the foreground.

[0080] The method further includes, for each tracked object in the scene, identifying S806 whether the tracked object has an uncertain object type status based on analysis of the first algorithm. The uncertain object type status may be based on an associated list of indicators.

[0081] The method further includes checking a background model to determine an object type status of the tracked object in the first image if the object has an uncertain object type status S808. The background model may be used to determine that the object is not an object of the first object type if the background model defines that image data corresponding to the object in the last image of the video sequence belongs to the background in the first image, and to determine that the object is an object of the first object type if the background model defines the image data as belonging to the foreground in the first image.

[0082] In some examples, if the object does not have an uncertain object type status, then the analysis of the object in the first image performed by the first algorithm is relied upon S810. This may mean determining that the object is the first object type when the first algorithm determines that the object is the first object type, and determining that the object is not the first object type when the first algorithm determines that the object is not the first object type.

[0083] In some examples, the method includes masking or highlighting S812, in the first image, the object determined to be of the first object type.

[0084] The methods illustrated in Figures 7-8, as well as any other methods or functionality described herein, may be stored as instructions on a non-transitory computer-readable storage medium such that the methods are performed when the instructions are executed on a device or system having processing capabilities. Such a device or system may comprise one or more processors. Thus, the features and methods described herein may be advantageously implemented in one or more computer programs executable on a programmable system that may include at least one programmable processor coupled to receive data and instructions from and transmit data and instructions to a data storage system, at least one input device such as a camera for capturing image frames / video sequences, and at least one output device such as a display for displaying the potentially masked or highlighted image described herein. Processors suitable for executing a program of instructions include, by way of example, both general-purpose and special-purpose microprocessors, as well as the sole processor or one or more processors or cores of any type of computer. The processor may be supplemented by or incorporated in an ASIC (Application Specific Integrated Circuit).

[0085] The above embodiments should be understood as illustrative examples of the present invention. Further embodiments of the present invention are envisioned. For example, the techniques described herein may be employed in any suitable object classification system, for example, used in autonomous vehicles, sports analysis, surveillance and security, and weather forecasting. It should be understood that any feature described in connection with any one embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may be employed without departing from the scope of the present invention, as defined in the appended claims.

[0086] Enumerated List of Embodiments A. A method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the method comprising: For each image frame of the plurality of image frames, analyzing the image frame using a first algorithm to identify objects of the first object type in the image frame, the first algorithm calculating for each region of a plurality of regions in the scene a probability that image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether the image data is part of an object of the first object type, and storing an indicator of a difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list of indicators associated with the region; detecting an object of a first object type in a first image frame, the first image frame being a last image frame of a plurality of image frames; and detecting includes: providing a background model of the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data corresponding to the region in a first image frame belongs to the background or the foreground, the background model being defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames; for each region of the scene, identifying whether image data corresponding to the region has an uncertain object type status in the first image based on an associated list of indicators; Upon identifying that the image data has an uncertain object type status, determining that the image data is not part of an object of the first object type when the background model defines the image data as belonging to a background in the first image; determining that the image data is part of an object of the first object type when the background model defines the image data as belonging to a foreground in the first image; A method comprising:

[0087] B. Identifying the image data as not having uncertain object type status determining that the image data is part of an object of the first object type when the first algorithm determines that the image data is part of an object of the first object type; determining that the image data is not part of an object of the first object type when the first algorithm determines that the image data is not part of an object of the first object type; The method of embodiment A, further comprising:

[0088] C. Identifying whether the image data has an uncertain object type status in the first image includes: determining whether an absolute value of a difference between a probability that the image data is part of an object of the first object type and a probability threshold exceeds a threshold difference based on the associated list of indicators; determining that the image data does not have uncertain object type status if the absolute value of the difference exceeds a threshold difference; if the difference does not exceed a threshold distance, determining a distribution measure between the indicators in the associated list of indicators indicating a positive difference and the indicators in the associated list of indicators indicating a negative difference; determining that the image data has an uncertain object type status if the distribution measure indicates a mixture of positive and negative differences that are within the uncertainty range, and otherwise determining that the image data does not have an uncertain object type status. The method of any one of embodiments A to B, comprising:

[0089] D. Identifying whether the image data has an uncertain object type status in the first image includes: determining a distribution measure between indicators in the associated list of indicators showing a positive difference and indicators in the associated list of indicators showing a negative difference; determining that the image data has an uncertain object type status if the distribution measure indicates a mixture of positive and negative differences that are within the uncertainty range, and otherwise determining that the image data does not have an uncertain object type status. The method of any one of embodiments A to B, comprising:

[0090] E. The method of embodiment D, wherein the list of indicators is a list of binary values, a positive difference being indicated by a first value of the binary values ​​and a negative difference being indicated by a second value of the binary values.

[0091] F. The method of any one of embodiments C to E, wherein the uncertainty range indicates a mix between a 20 / 80 to 80 / 20 distribution of indicators indicating positive and negative differences.

[0092] G. The method of any one of embodiments A to E, wherein the first algorithm includes a feature extraction process for extracting features of objects in the image and representing them as vectors of numbers.

[0093] H. The method of any one of embodiments A to G, wherein the second algorithm is a motion-based background detection algorithm.

[0094] I. The method of any one of embodiments A to H, wherein the list of indicators associated with the region is a FIFO list having 5 to 15 values.

[0095] J. The method of any one of embodiments A to I, further comprising masking or highlighting image data in the first image that is determined to be part of an object of the first object type.

[0096] K. The method of any one of embodiments A to J, wherein the method is implemented in a camera and the video sequence is part of a live video stream captured by the camera.

[0097] L. A method for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the scene including a plurality of objects tracked in the plurality of image frames, the method comprising: For each image frame of the plurality of image frames, Analysing the image frames using a first algorithm to identify objects of a first object type in the image frames, the first algorithm calculating, for each object of a plurality of objects tracked in the image frames, a probability that the object is an object of the first object type, the first algorithm having a probability threshold for determining whether the object is an object of the first object type, and storing an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list of indicators associated with the object; detecting an object of a first object type in a first image frame, the first image frame being a last image frame of a plurality of image frames; and detecting includes: providing a background model of the video sequence, the background model defining, for each region of a plurality of regions in the scene, whether image data corresponding to the region in a first image frame belongs to the background or the foreground, the background model being defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames; For each object of the plurality of objects tracked in the first image, identifying whether the object has an uncertain object type status based on a list of indicators associated with the object; Upon identifying an object as having an uncertain object type status, determining image data corresponding to the object in the first image; determining that the object is not an object of the first object type if the background model defines the image data as belonging to a background in the first image; determining that the object is an object of the first object type if the background model defines the image data as belonging to a foreground in the first image; A method comprising:

[0098] M. The method of embodiment L, in which each of the tracked objects is located more than a threshold distance from the camera that captured the video sequence of the scene.

[0099] N. Identifying an object as not having uncertain object type status determining that the object is an object of the first object type when the first algorithm determines that the object is an object of the first object type; determining that the object is not an object of the first object type when the first algorithm does not determine that the object is an object of the first object type; The method of any one of embodiments L to M, further comprising:

[0100] O. Identifying whether an object has an uncertain object type status comprises: determining whether an absolute value of a difference between a probability that the object is an object of the first object type in the first image and a probability threshold exceeds a threshold difference based on the associated list of indicators; determining that the object does not have an uncertain object type status if the absolute value of the difference exceeds a threshold difference; if the difference does not exceed a threshold distance, determining a distribution measure between the indicators in the associated list of indicators indicating a positive difference and the indicators in the associated list of indicators indicating a negative difference; determining that the object has an uncertain object type status if the distribution measurement shows a mixture of positive and negative differences that are within the uncertainty range, and otherwise determining that the object does not have an uncertain object type status. The method of any one of embodiments L to N, comprising:

[0101] P. Identifying whether an object has an uncertain object type status comprises: determining a distribution measure between indicators in the associated list of indicators showing a positive difference and indicators in the associated list of indicators showing a negative difference; determining that the object has an uncertain object type status if the distribution measurement shows a mixture of positive and negative differences that are within the uncertainty range, and otherwise determining that the object does not have an uncertain object type status. The method of any one of embodiments L to N, comprising:

[0102] Q. The method of embodiment P, in which the list of indicators is a list of binary values, a positive difference being indicated by a first value of the binary values ​​and a negative difference being indicated by a second value of the binary values.

[0103] R. The method of any one of embodiments O to Q, wherein the uncertainty range indicates a mixture between a 20 / 80 to an 80 / 20 distribution of indicators indicating positive and negative differences.

[0104] S. The method of any one of embodiments L to R, wherein the first algorithm includes a feature extraction process for extracting features of objects in the image and representing them as vectors of numbers.

[0105] T. The method of any one of embodiments L to S, wherein the second algorithm is a motion-based background detection algorithm.

[0106] U. The method of any one of embodiments L to T, wherein the list of indicators associated with the object is a FIFO list having 5 to 15 values.

[0107] V. The method of any one of embodiments L-U, further comprising masking or highlighting, in the first image, objects determined to be of the first object type.

[0108] X. The method of any one of embodiments L to V, wherein the method is implemented in a camera and the video sequence is part of a live video stream captured by the camera.

[0109] Y. A non-transitory computer-readable storage medium storing instructions for performing a method according to any one of claims A through K or a method according to any one of claims L through X when executed on a device having processing capability.

[0110] Z. A system for detecting an object of a first object type in a first image frame of a plurality of image frames in a video sequence capturing a scene, the system comprising: one or more processors; one or more non-transitory computer readable media storing computer executable instructions that, when executed by one or more processors, cause the system to perform the method of any one of claims A through K or any one of claims L through X; A system comprising:

Claims

1. A method (700) for detecting an object (102) of a first object type in a first image frame (100c) among a plurality of image frames (100a - c) in a video sequence capturing a scene, the method comprising: For each image frame of the plurality of image frames, Analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates, for each of a plurality of regions in the scene, the probability that the image data corresponding to the region is part of an object of the first object type in the image frame, the first algorithm having a probability threshold for determining whether the image data is part of the object of the first object type, and storing an indicator of the difference between the probability that the image data is part of an object of the first object type and the probability threshold in a list (302, 402) of indicators associated with the region, identifying the object of the first object type in the image frame (S702); Detecting an object of the first object type in the first image frame, wherein the first image frame is the last image frame among the plurality of image frames, detecting the object of the first object type in the first image frame; Including, the detecting comprising: Providing a background model (300) of the video sequence, wherein the background model defines, for each of the plurality of regions in the scene, whether the image data corresponding to the region in the first image frame belongs to the background (302) or the foreground (304, 306), and the background model is defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames, providing the background model of the video sequence (S704); For each region of the scene, based on a plurality of indicators in the associated list of indicators including a separate indicator for each image frame of the plurality of image frames, identifying whether the image data corresponding to the region in the first image frame has an uncertain object type status (S706); When it is identified that the image data has an uncertain object type status, when the background model defines the image data as belonging to the background in the first image frame, it is determined that the image data is not part of an object of the first object type, when the background model defines the image data as belonging to the foreground in the first image frame, determining that the image data is part of an object of the first object type (S708); and A method including the above. **Claim 2** Identifying the image data as not having an uncertain object type status (S701); when the first algorithm determines that the image data is part of the object of the first object type, determining that the image data is part of the object of the first object type; when the first algorithm determines that the image data is not part of the object of the first object type, determining that the image data is not part of the object of the first object type; and The method according to claim 1, further including the above. **Claim 3** The step of identifying whether the image data has an uncertain object type status in the first image frame includes: using an indicator from a list of relevant indicators indicating the difference between the probability that the image data is part of an object of the first object type and the probability threshold, to determine whether the absolute value of the difference between the probability that the image data is part of an object of the first object type and the probability threshold exceeds a threshold difference; when the absolute value of the difference exceeds the threshold difference, determining that the image data does not have an uncertain object type status; when the difference does not exceed the threshold difference, applying a statistical method to determine a distribution measurement value between an indicator in the list of relevant indicators indicating a positive difference and an indicator in the list of relevant indicators indicating a negative difference; when the distribution measurement value indicates a mixture of positive and negative differences included within an uncertainty range, determining that the image data has an uncertain object type status, otherwise determining that the image data does not have an uncertain object type status. The method according to claim 1, comprising, wherein the uncertainty range includes an upper threshold value and a lower threshold value, and the upper threshold value and the lower threshold value define a statistical measure according to the statistical method.

4. The step of identifying whether the image data has an uncertain object type status in the first image frame comprises: determining a distribution measurement value between the indicator in the list of associated indicators indicating a positive difference and the indicator in the list of associated indicators indicating a negative difference; if the distribution measurement value indicates a mixture of positive and negative differences included within the uncertainty range, determining that the image data has an uncertain object type status, and otherwise determining that the image data does not have an uncertain object type status; The method according to claim 1, comprising.

5. The method according to claim 4, wherein the list of indicators is a list of binary values, a positive difference is indicated by a first value of the binary value, and a negative difference is indicated by a second value of the binary value.

6. The method according to claim 3, wherein the uncertainty range indicates a mixture between a 20 / 80 distribution and an 80 / 20 distribution of the indicator indicating a positive difference and the indicator indicating a negative difference.

7. The method according to claim 1, wherein the first algorithm includes a feature extraction process for extracting features of objects in the image and representing them as a vector of numbers.

8. The method according to claim 1, wherein the second algorithm is a motion-based background detection algorithm.

9. The method according to claim 1, wherein the list of indicators associated with the region is a FIFO list having 5 to 15 values.

10. The method according to claim 1, further comprising masking or highlighting (S712) the image data determined to be part of an object of the first object type in the first image frame.

11. The method according to claim 1, implemented in a camera, wherein the video sequence is part of a live video stream captured by the camera.

12. A method (800) for detecting an object (202, 204) of a first object type in a first image frame (200c) among a plurality of image frames (200a-c) in a video sequence capturing a scene, wherein the scene includes a plurality of objects (202, 204, 218, 220, 222) tracked in the plurality of image frames, and the method comprises: For each image frame of the plurality of image frames, Analyzing the image frame using a first algorithm to identify an object of the first object type in the image frame, wherein the first algorithm calculates a probability that each object of the plurality of objects tracked in the image frame is an object of the first object type, the first algorithm has a probability threshold for determining whether the object is an object of the first object type, and storing an indicator of a difference between the probability that the object is an object of the first object type and the probability threshold in a list (302, 402) of indicators associated with the object, identifying the object of the first object type in the image frame (S802); Detecting an object of the first object type in the first image frame, wherein the first image frame is the last image frame among the plurality of image frames, detecting the object of the first object type in the first image frame Including, the detecting comprising: Providing a background model (400) of the video sequence, wherein the background model defines, for each region of a plurality of regions in the scene, whether image data corresponding to the region in the first image frame belongs to the background (400) or the foreground (404, 406, 408, 410), and the background model is defined by applying a second algorithm different from the first algorithm to at least some of the plurality of image frames, providing the background model of the video sequence (S804); For each object of the plurality of objects tracked in the first image frame, identifying whether the object has an uncertain object type status based on the list of indicators associated with the object, the list including a separate indicator for each image frame of the plurality of image frames (S806); Identify that the object has an uncertain object type status (S808), Determine the image data corresponding to the object in the first image frame, When the background model defines the image data as belonging to the background, determine that the object is not an object of the first object type, When the background model defines the image data as belonging to the foreground, determine that the object is an object of the first object type A method comprising.

13. The method according to claim 12, wherein each of the tracked objects is located beyond a threshold distance (216) from the camera that captured the video sequence of the scene.

14. A non-transitory computer-readable storage medium storing instructions for implementing the method according to claim 1 or the method according to claim 12 when executed on a device having processing capabilities.

15. A system for detecting an object of a first object type in a first image frame among a plurality of image frames in a video sequence that captured a scene, the system comprising: One or more processors; One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the system to perform the method according to claim 1 or the method according to claim 12 A system comprising.

Images