Detection of reflections of objects in a sequence of image frames

Abstract

The present disclosure relates to detection of reflections of objects in a sequence of image frames. In particular, techniques are provided for detecting reflections of objects in a sequence of image frames. Objects of a given type are detected in the sequence of image frames and a detection score is determined for each detected object. A distance ratio is determined between unfiltered object location indicators identified for each detected object. A displacement factor is determined for each detected object between a current location of the detected object and a previous location of the detected object. In response to finding a match between a normalized size of a detected object and a normalized movement, it is determined that one of two detected objects having a lower detection score is a reflection of the other of the two detected objects.

Description

Technical Field

[0001] The embodiments presented herein relate to methods, controllers, computer programs, and computer program products for detecting reflections of objects in a sequence of image frames. Background Technology

[0002] Generally speaking, object detection is a computer technology related to computer vision and image processing used to detect instances of semantic objects (such as people, animals, vehicles, etc.) belonging to a specific category. Some object detection algorithms generate keypoints or other types of object location indicators. These points or indicators typically represent points of interest of the target object being tracked from one image frame to the next.

[0003] There are scenarios or environments where the object detection mentioned above becomes challenging. Take urban environments as an example, which are characterized by numerous reflective surfaces. These surfaces can be mirrors, but they can also be thin sheets made of glass or other types of reflective materials. These surfaces often pose problems for object detection algorithms. In some cases, both the detected object and its reflection will be counted. In applications where counting detected objects is required, this can be problematic, causing some detected objects to be counted twice. In other cases, reflections may be completely missed. This can also be a problem in applications where detected objects are masked, but reflections are not.

[0004] US2020 / 0175694A1 discloses an information processing apparatus including a designation circuit and a counting circuit. The designation circuit designates two or more movable objects corresponding to the same real movable object among the movable objects, based on the similarity of the rates of positional change of multiple movable objects in an image. The counting circuit counts the number of real movable objects corresponding to the movable objects based on the designation result of the designation circuit.

[0005] However, in practice, a reflection is not exactly equal to the tracked object it reflects. A reflection may appear smaller or larger than the tracked object. Furthermore, reflections from translucent surfaces can result in less detail of the reflection in the image frame compared to the tracked object. Additionally, a reflection may only include a portion of the tracked object. This can lead to incorrect decisions by the designated circuitry in the information processing apparatus described in US2020 / 0175694A1 regarding whether two or more movable objects correspond to the same real movable object. Summary of the Invention

[0006] One objective of the embodiments described herein is to address the aforementioned problems and provide improved detection of objects on reflective surfaces.

[0007] Therefore, according to the first aspect, a method for detecting reflections of objects in an image frame sequence is proposed. The method is executed by a controller. The method includes detecting objects of a given type in the image frame sequence and determining a detection score for each detected object. The method includes, for each detected object, determining a distance ratio between unfiltered object position indicators identified for the detected object. Unfiltered object position indicators are identified in the current image frame of the image frame sequence. The distance ratio of the detected objects defines the normalized size of the detected objects. The method includes determining a displacement factor between the current position of the detected object in the current image frame and the previous position of the detected object in a previous image frame of the image frame sequence for each detected object. The displacement factor of the detected object defines the normalized movement of the detected object. The method includes, in response to finding a match between the normalized size and normalized movement of two detected objects, determining that the detected object with the lower detection score is a reflection of the other detected object.

[0008] According to a second aspect, a controller for detecting object reflections in a sequence of image frames is provided. The controller includes processing circuitry. The processing circuitry is configured to cause the controller to detect objects of a given type in the sequence of image frames and to determine a detection score for each detected object. The processing circuitry is configured to cause the controller, for each detected object, to determine a distance ratio between unfiltered object position indicators identified for the detected object. Unfiltered object position indicators are identified in the current image frame of the image frame sequence. The distance ratio of the detected objects defines the normalized size of the detected object. The processing circuitry is configured to cause the controller, for each detected object, to determine a displacement factor between the current position of the detected object in the current image frame and the previous position of the detected object in a previous image frame of the image frame sequence. The displacement factor of the detected object defines the normalized movement of the detected object. The processing circuitry is configured to cause the controller, in response to finding a match between the normalized size and normalized movement of two detected objects, to determine that the detected object with the lower detection score is a reflection of the other detected object.

[0009] According to the third aspect, a video surveillance system is provided. This video surveillance system includes a controller according to the second aspect and a camera for capturing sequences of image frames.

[0010] According to the fourth aspect, a computer program for detecting reflections of objects in a sequence of image frames is provided, the computer program including computer program code that, when run on a controller, causes the controller to perform the method according to the first aspect.

[0011] According to the fifth aspect, a computer program product is provided, including the computer program according to the fourth aspect and a computer-readable storage medium storing the computer program. The computer-readable storage medium may be a non-transitory computer-readable storage medium.

[0012] Advantageously, these aspects provide computationally efficient and accurate object detection in scenes or environments with reflective surfaces.

[0013] Other objects, features, and advantages of the appended embodiments will become apparent from the following detailed disclosure, the appended dependent claims, and the accompanying drawings.

[0014] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field, unless otherwise expressly defined herein. All references to “a / the element, device, component, means, module, step, etc.” should be interpreted as referring to at least one instance of an element, device, component, means, module, step, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed, unless expressly stated. Attached Figure Description

[0015] The concept of the invention will now be described by way of example with reference to the accompanying drawings, in which:

[0016] Figure 1 This is a schematic diagram of a video surveillance system according to an embodiment;

[0017] Figure 2 An image frame including a detected object and a reflection of the detected object is illustrated according to an embodiment;

[0018] Figure 3 The illustration schematically shows a detected object and a bounding box surrounding the detected object according to an embodiment;

[0019] Figure 4 The illustration schematically shows the detected object and key points of the detected object according to an embodiment;

[0020] Figure 5 The illustration schematically shows key points of a detected object and key points of the reflection of the detected object according to an embodiment;

[0021] Figure 6 This is a flowchart of a method according to an embodiment;

[0022] Figure 7 An image frame after the first action is performed, according to an embodiment, is illustrated schematically;

[0023] Figure 8 An image frame after the second action is performed, according to an embodiment, is illustrated schematically;

[0024] Figure 9 This is a schematic diagram of the functional units of the controller according to an embodiment; and

[0025] Figure 10 An example of a computer program product including a computer-readable storage medium according to an embodiment is shown. Detailed Implementation

[0026] The present invention will now be described more fully with reference to the accompanying drawings, which illustrate specific embodiments of the invention. However, the concept of the invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Throughout the description, the same reference numerals refer to the same elements. Any step or feature indicated by dashed lines should be considered optional.

[0027] Figure 1 This is a schematic diagram of a video surveillance system 100 to which the embodiments described herein can be applied. Camera 120 is configured to capture image frames within the field of view (FoV) 130 of scene 140. Figure 1 In the illustrative example, scene 140 includes an object 150-1 in human form. Camera 120 includes a controller 110, which is paired with, integrated with, or at least operatively connected to the controller 110. It is assumed that object 150-1 represents a target object captured by camera 120 and analyzed by controller 110, to be tracked frame by frame. For this purpose, controller 110 may implement an object detection algorithm.

[0028] The embodiments disclosed herein relate to techniques for detecting reflections of object 150-1 in a sequence of image frames. To obtain such techniques, a controller 110, a method executed by the controller 110, and a computer program product including code, such as in the form of a computer program, are provided, which, when run on the controller 110, cause the controller 110 to execute the method.

[0029] Figure 2Image frame 200 is schematically shown. Image frame 200 depicts an object 150-1 in human form, as in scene 140. Object 150-1 is located in image frame 200 at a position marked as reference numeral 220-1. A reflective surface 210 is also depicted in image frame 200. Reflective surface 210 is located in image frame 200 at a position marked as reference numeral 230. Reflective surface 210 also makes another object 150-2 visible. Object 150-2 is a reflection of object 150-1 because object 150-1 is reflected by reflective surface 210. Object 150-2 is located in image frame 200 at a position marked as reference numeral 220-2.

[0030] The inventors recognized that the common feature of objects 150-1 and 150-2 is their object position indicators. (This document references...) Figure 3 and Figure 4 They schematically depict human-shaped objects 150-1, as in scene 140. Figure 3 and Figure 4 The diagram shows different types of object location indicators. Figure 3 In the image, object 150-1 is surrounded by bounding box 310. Bounding box 310 is defined by two corner points 320a and 320b and a center point 330. Center point 330 can represent the position 220-1 of object 150-1. For example, corner points 320a and 320b and center point 330 can be CenterNet keypoints. For example, the bounding box can be a CenterNet bounding box. In this respect, the object position indicator can be defined by two or more bounding boxes. That is, two or more bounding boxes can be used to surround the detected object 150-1. That is, a first bounding box can surround the first part of the detected object 150-1 (such as the head), a second bounding box can surround the second part of the detected object 150-1 (such as the torso), and so on. Figure 4 In this context, object 150-1 is covered by object location indicators 410-1 to 417b-1 listed in Table 1. These object location indicators can be considered as simplified versions of COCO keypoints or MediaPipe pose keypoints.

[0031]

[0032] Table 1: Correspondence between reference numerals and object parts

[0033] The reflected object 150-2 will have the same object location indicator settings, but will be reflected. By analyzing the object location indicator, it is possible to determine the object-reflection pair at the object level, i.e., a tracked object 150-1 and its reflected corresponding object 150-2, or even at the object location indicator level. (References) Figure 5 The schematic diagram shows the object location indicators 410-1 to 417b-1 listed in Table 1 for object 150-1 (not shown). Figure 5 It also indicates the position 220-1 of object 150-1. Figure 5 Object position indicators 410-2 to 417b-2 for object 150-2 (not shown) are also shown. Object 150-2 is a reflection of object 150-1 in the reflective surface 210 at position 230. Object position indicators 410-2 to 417b-2 correspond to the object position indicators listed in Table 1. That is, object position indicator 410-2 indicates the head of object 150-2, and object position indicator 417b-2 indicates the left foot of object 150-2.

[0034] Figure 6 This is a flowchart illustrating an embodiment of a method for detecting the reflection of object 150-1 in image frame sequences 200, 800, and 900. The method is executed by controller 110. The method is advantageously provided as computer program 1120.

[0035] S102: Controller 110 detects objects 150-1 and 150-2 of a given type in image frame sequences 200, 800, and 900. Controller 110 determines a detection score for each detected object 150-1 and 150-2.

[0036] In some examples, in addition to the detection score determined for each detected object 150-1, 150-2 as a whole, a detection score is also determined for each object location indicator identified for each detected object 150-1, 150-2. The detection scores of individual object location indicators only become relevant if the detection scores of the detected objects 150-1, 150-2 are above some threshold. Conversely, in some aspects, the detection score of a given detected object 150-1, 150-2 is a function of the detection scores of all unfiltered object location indicators identified for that given detected object 150-1, 150-2.

[0037] S104: For each detected object 150-1, 150-2, the controller 110 determines the distance ratio between the unfiltered object location indicators identified for the detected objects 150-1, 150-2. Unfiltered object location indicators are identified in the current image frames 200, 800, 900 within the sequence of image frames 200, 800, 900. The distance ratio of the detected objects 150-1, 150-2 defines the normalized size of the detected objects 150-1, 150-2.

[0038] Since the size of the detected object is determined for the entire detected object, it is considered to be the normalized size of the detected object.

[0039] S106: For each detected object 150-1, 150-2, the controller 110 determines a displacement factor between the current positions 220-1, 220-2 of the detected objects 150-1, 150-2 in the current image frames 200, 800, 900 and the previous positions of the detected objects 150-1, 150-2 in the image frame sequence 200, 800, 900. The displacement factor of the detected objects 150-1, 150-2 defines the normalized movement of the detected objects 150-1, 150-2.

[0040] Therefore, regarding previous image frames in the image frame sequence 200, 800, and 900, historical data is used by the controller 110 to obtain position information of objects detected in previous image frames. A measure of the movement of the detected object can be obtained based on how much it has moved from the previous image frame to the current image frame. Since the movement of the detected object is determined for the entire detected object, this movement is considered a normalized movement of the detected object.

[0041] Then, the normalized size and normalized movement of two detected objects 150-1 and 150-2 can be compared with each other. That is, the normalized size of one detected object 150-1 and 150-2 is compared with the normalized size of the other detected object 150-1 and 150-2, and the normalized movement of one detected object 150-1 and 150-2 is compared with the normalized movement of the other detected object 150-1 and 150-2. Furthermore, in this respect, in general, normalized size and normalized movement represent that a particular given metric (the size of the normalized size and the movement of the normalized movement) has been scaled or otherwise recalculated to a common scale. As a non-limiting example, suppose that object 150-1 and its reflection are seen at different distances from camera 120, the effect will be that object 150-1 and its reflection appear to have different sizes from each other; the object closest to the camera appears larger, and vice versa. By using normalized size, object 150-1 and its reflection will be scaled or otherwise recalculated to a common size scale, where the effect will be taken into account so that object 150-1 and its reflection will appear to be the same size. Similarly, under the same assumptions, the effect will be that object 150-1 and its reflection will appear to move at different speeds; the object closest to the camera will appear to move faster, and vice versa. By using normalized motion, object 150-1 and its reflection will be scaled or otherwise recalculated to a common motion scale, where the effect will be taken into account so that object 150-1 and its reflection will appear to move at the same speed.

[0042] If the normalized size and normalized movement of two detected objects 150-1 and 150-2 are similar, it can be concluded that the two detected objects 150-1 and 150-2 have similar size and similar movement. This means that one of the detected objects 150-1 and 150-2 is a reflection of the other detected object.

[0043] S110: In response to the detection of a match between the normalized size and normalized movement of two detected objects 150-1, 150-2 among the detected objects, the controller 110 determines that the detected object with the lower detection score of the two detected objects 150-1, 150-2 is a reflection of the other detected object of the two detected objects 150-1, 150-2.

[0044] Therefore, by considering the detection scores of the two detected objects 150-1 and 150-2, the detected object 150-2 with a relatively low detection score is classified as a reflection of the detected object 150-1 with a relatively high detection score.

[0045] An embodiment of further details of the detection of the reflection of object 150-1 in image frame sequences 200, 800, 900 performed by controller 110 will now be disclosed.

[0046] As disclosed above, controller 110 detects objects 150-1 and 150-2 of a given type in image frame sequences 200, 800, and 900. In this respect, controller 110 can detect objects 150-1 and 150-2 of different types. In some non-limiting examples, the given type is a person, an animal, or a vehicle, such as a car or a truck. Therefore, controller 110 may have been trained and thus configured to detect objects 150-1 and 150-2 of a given type.

[0047] As disclosed above, controller 110 determines, for each detected object 150-1, 150-2, the distance ratio between unfiltered object location indicators identified for the detected objects 150-1, 150-2. Generally, an unfiltered object location indicator is an object location indicator determined without first filtering out any objects already detected with low detection scores. That is, an unfiltered object location indicator refers to the object location indicator before thresholding is performed to mitigate false positives in the scene. Because reflections typically have lower detection scores than their non-reflective counterparts, using unfiltered object location indicators ensures that detected objects 150-2, as reflections, are not missed. In other words, some unfiltered object location indicators may belong to objects that would otherwise be filtered out or discarded due to low detection scores. Therefore, in some embodiments, unfiltered object location indicators are determined for all detected objects 150-1, 150-2, without any detected objects 150-1, 150-2 (because their detection scores are below the detection threshold) being filtered out.

[0048] Furthermore, different types of object location indicators can exist in this regard.

[0049] In some non-restrictive examples, each object location indicator is a common object COCO in the environment, keypoints 410-1:417b-1, 410-2:417b-2, or a MediaPipe pose keypoint. In this respect, the object location indicator does not need to be defined by all COCO keypoints or MediaPipe pose keypoints; rather, a subset of these keypoints can be used as object location indicators. As mentioned above, this is in... Figure 4As shown in the image.

[0050] In other non-limiting examples, the object location indicators are the corner points 320s and 320b and the center point 330 of the bounding box 310, such as the CenterNet bounding box. As mentioned above, this is in Figure 3 As shown in the image.

[0051] As disclosed below, in response to the discovery of a match between the normalized size and / or normalized movement of two detected objects 150-1, 150-2 among the detected objects, it is determined that one of the two detected objects 150-1, 150-2 is a reflection of the other detected object among the two detected objects 150-1, 150-2. Therefore, in some embodiments, the controller 110 is configured to perform step S108 (optionally).

[0052] S108: Controller 110 detects a match between the normalized size and / or normalized movement of two detected objects 150-1 and 150-2 among the detected objects.

[0053] The controller 110 can discover a match between the normalized size and / or normalized movement of two detected objects 150-1 and 150-2 among the detected objects in different ways. In some aspects, this match discovery is performed by comparing the normalized size and / or normalized movement of the first detected object 150-1 with the normalized size and / or normalized movement of the second detected object 150-2. In particular, in some embodiments, the controller 110 is configured to perform (optionally) steps S108a and / or S108b as part of step S108.

[0054] S108a: Controller 110 confirms that the size difference between the normalized size of one of the two detected objects 150-1 and 150-2 and the normalized size of the other detected object is less than a size threshold.

[0055] Therefore, the smaller the size difference between the normalized size of one of the two detected objects 150-1 and 150-2 and the normalized size of the other detected object 150-1 and 150-2, the higher the chance of a match.

[0056] S108b: Controller 110 confirms that the movement difference between the normalized movement of one of the two detected objects 150-1 and 150-2 and the normalized movement of the other detected object is less than the movement threshold.

[0057] Therefore, the smaller the movement difference between the normalized movement of one of the two detected objects 150-1 and 150-2 and the normalized movement of the other detected object 150-1 and 150-2, the higher the chance of a match.

[0058] In some respects, matching is performed at the object position indicator level. That is, instead of comparing only the normalized size and / or normalized movement of the first detected object 150-1 with the normalized size and / or normalized movement of the second detected object 150-2, a comparison is made between the object position indicator pairs of one of the two detected objects 150-1, 150-2 and the object position indicator pairs of the other detected object of the two detected objects 150-1, 150-2 (in terms of size and / or movement).

[0059] In other words, for illustrative purposes, assume that the two detected objects 150-1 and 150-2 represent a person, and that the knees of both legs are detectable and represented by object position indicators in the two detected objects 150-1 and 150-2. A first comparison can then be made between the normalized distance between the knees of the detected object based on the object position indicator of one of the two detected objects 150-1 and 150-2, and the normalized distance between the knees of the detected object based on the object position indicator of the other detected object 150-1 and 150-2. Furthermore, a second comparison can be made between the normalized movement of the knees of the detected object 150-1 and 150-2 as given by the object position indicator and the normalized movement of the knees of the detected object 150-1 and 150-2 as given by the object position indicator. Further such comparisons in size and / or movement can be made for other parts of the detected objects to accumulate the matching scores of the two detected objects 150-1 and 150-2.

[0060] In some aspects, when it is determined that one of the two detected objects 150-1, 150-2 is a reflection of the other detected object, the controller 110 performs certain actions. Therefore, in some embodiments, the controller 110 is configured to perform step S112 (optionally).

[0061] S112: Controller 110 performs an action on detection object 150-2, which is a reflection of another detection object 150-1.

[0062] In step S112, there are different types of actions that can be performed by controller 110. In some non-limiting examples, the action involves applying shielding 810-2 to the detected object 150-2 as a reflection of another detected object 150-1. (Intermediate references) Figure 7 The schematic diagram illustrates image frame 800. Image frame 800 describes how the first shield 810-1 is placed... Figure 2 The location of object 150-1, and how to place the second shield 810-2. Figure 2 The location where object 150-2 appears. In some non-limiting examples, the action involves filtering out detected object 150-2 as a reflection of another detected object 150-1, lowering the detection score threshold of detected object 150-2 as a reflection of another detected object 150-1, etc. (Intermediate references in this document) Figure 8 The schematic diagram illustrates image frame 900. Image frame 900 describes, as follows: Figure 2 Object 150-1 is shown in the image frame 900, but object 150-2 has been filtered out so that it does not appear in the image frame 900. In this respect, although object 150-2, representing a reflection, is shown as being filtered out from the image frame, note that filtering usually refers to filtering out detected object 150-2 from some count or other type of calculation so that object 150-1 is not counted twice.

[0063] In some aspects, such as Figure 2 As shown, the reflection is caused by a reflective surface 210 placed between the two detected objects 150-1, 150-2. Generally, the reflective surface 210 is positioned at the midpoint between positions 220-1, 220-2 of the two detected objects 150-1, 150-2. Therefore, the knowledge of the positions 220-1, 220-2 of the two detected objects 150-1, 150-2 is used to determine the position 230 of the reflective surface 210. Therefore, in some embodiments, the controller 110 is configured to perform (optionally) step S114.

[0064] S114: Controller 110 determines the position 230 of reflective surface 210 such that one of the detected objects 150-1 and 150-2 becomes a reflection of the other detected object 150-1 and 150-2. The position 230 of reflective surface 210 is determined to be the midpoint between the positions 220-1 and 220-2 of the two detected objects 150-1 and 150-2.

[0065] In some embodiments, knowing the position 230 of the reflective surface 210 is used to detect other objects in image frame sequences 200, 800, 900. Since the position 230 of the reflective surface 210 can be used as a reference point, or a reference line, or even a reference surface, knowing the position 230 can increase the chance of detecting reflective objects under challenging lighting conditions in future image frames of the same scene, or at least in scenes where the reflective surface 210 is still present.

[0066] It should be noted that when matching by referencing the normalized dimensions of objects 150-1 and 150-2, compared to the object position indicators 410-1 to 417b-1 of object 150-1, it is not necessary to consider that the object position indicators 410-2 to 417b-2 of object 150-2 are mirror images. For the same reason, when matching by referencing the normalized movement of objects 150-1 and 150-2, since the reflective surface 210 is positioned relative to the detected objects 150-1 and 150-2, the normalized movement of object 150-1 will be the same as the normalized movement of object 150-2, except for possible sign variations. For example, using, as Figure 2 The placement of the reflective surface 210 in the image, if object 150-1 appears to be facing... Figure 2 Moving the object to the left in the middle makes object 150-2 appear to be facing... Figure 2 Rightward movement within the context. To address this potential issue, normalized movement can be represented solely by its magnitude, regardless of its direction. Furthermore, for any match performed at the object position indicator level, reflection should be considered when the match involves any object position indicator, for which each object has a right object position indicator and a left object position indicator, such as object position indicators 412a-1, 412b-1, etc.

[0067] The method and controller 110 disclosed so far can be used as part of a technique to improve the detection of people and other objects 150-1 in a scene 140 with a glass surface. The glass surface can be reflective and transmissive. The reflected image may be darker than the actual image of the person or object. This is also true for people or other objects 150-1 visible through the glass, especially under certain lighting conditions. As disclosed, if the purpose of detecting object 150-1 is to mask it, there is a risk that the mirror image of object 150-1 remains unmasked and may be identified if the threshold used for detection is too strict. Conversely, if the purpose of detecting object 150-1 is for counting, there is a risk of double-counting objects 150-1 if the detection threshold is too high.

[0068] Therefore, in some respects, it is possible to identify areas where both reflected and transmitted images may occur. Identifying these areas may be easier under specific lighting conditions, and knowledge gained under favorable conditions can be used under more challenging conditions. If a surface can be identified where reflections 150-1 are frequently found 150-2, but reflections are also frequently not found (e.g., keypoint mismatch), then the surface is likely both reflective and transmissive.

[0069] Illumination conditions can affect the degree of reflection and / or transmission of an object (150-1, 150-2). For example, studying surfaces with high reflectivity or translucency is generally relatively easy because it makes detection clearer and therefore more reliable. On the other hand, objects are often difficult to detect in dim lighting conditions. Therefore, information about illumination conditions can be used when determining whether an object is reflective. For example, a threshold can be lowered when dim lighting conditions are indicated. This also helps to adjust keypoint reflection algorithms. For example, if it is found that reflection is more likely than transmission in a given area of ​​the image, a more aggressive search for keypoint pairs can be performed in that given area of ​​the image. Furthermore, bright lighting conditions generally produce more reflective objects than transmissive objects. This knowledge can also be used to help the algorithm. Therefore, if the lighting conditions indicate a higher chance of reflection, a more aggressive search for keypoint pairs can be performed.

[0070] Possible methods will now be briefly described. Reflections can be detected using the methods described above. The number of reflections recorded in a given portion of scene 140 is saved over time. This can be viewed as generating a heatmap. If multiple recorded reflections exist, and there are multiple real targets for a given surface, this is marked in the heatmap. For privacy masking applications, static masking can be applied to that surface to ensure reflections are masked, even if they are below the current detection threshold. Alternatively, the masking threshold required to mask object 150-2 can be reduced in that area. For each image frame 200, whether detected objects 150-1 and 150-2 are reflections can be determined by finding reflection pairs in scene 140. Therefore, by constructing data over time using the probability of whether object 150-1 is a reflection, it is possible to predict the probability of whether new objects located in the same area of ​​image frames in the future are reflections.

[0071] Figure 9 The schematic illustration shows the components of the controller 110 according to an embodiment across multiple functional units. The processing circuitry 1010 uses components capable of executing computer programs stored in the product 1110 (such as…). Figure 10The processing circuitry 1010 may be provided as any combination of one or more suitable central processing units (CPUs), multiprocessors, microcontrollers, digital signal processors (DSPs), etc., containing software instructions (e.g., in the form of storage medium 1030). The processing circuitry 1010 may also be provided as at least one application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA).

[0072] Specifically, the processing circuit 1010 is configured to cause the controller 110 to perform a set of operations or steps, as disclosed above. For example, the storage medium 1030 may store the set of operations, and the processing circuit 1010 may be configured to retrieve the set of operations from the storage medium 1030 so that the controller 110 executes the set of operations. The set of operations may be provided as a set of executable instructions.

[0073] Therefore, the processing circuitry 1010 is thus arranged to perform the methods disclosed herein. The storage medium 1030 may also include permanent memory, which may be any one or a combination of magnetic storage, optical storage, solid-state storage, or even remotely mounted memory. The controller 110 may further include a communication interface 1020, which is configured at least to communicate with the camera 120 and may communicate with other functions, nodes, entities, and / or devices, such as functions, nodes, entities, and / or devices of a video surveillance system. Thus, the communication interface 1020 may include one or more transmitters and receivers, including analog and digital components. The processing circuitry 1010 controls the overall operation of the controller 110, for example, by sending data and control signals to the communication interface 1020 and the storage medium 1030, by receiving data and reports from the communication interface 1020, and by retrieving data and instructions from the storage medium 1030. Other components and related functions of the controller 110 are omitted to avoid obscuring the concepts presented herein.

[0074] Controller 110 may be provided as a standalone device or as part of at least one other device. For example, controller 110 and camera 120 may be part of video surveillance system 100. Alternatively, controller 110 may be integrated into camera 120. A first portion of the instructions executed by controller 110 may be executed in a first device, and a second portion of the instructions executed by controller 110 may be executed in a second device; the embodiments disclosed herein are not limited to any particular number of devices on which the instructions executed by controller 110 may be executed. Therefore, the methods according to the embodiments disclosed herein are applicable to execution by controller 110 residing in a cloud computing environment. Therefore, although Figure 9 A single processing circuit 1010 is shown, but the processing circuit 1010 can be distributed across multiple devices or nodes. This also applies to… Figure 10 Computer program 1120.

[0075] Figure 10 An example of a computer program product 1110 including a computer-readable storage medium 1130 is shown. On this computer-readable storage medium 1130, a computer program 1120 may be stored, which can cause processing circuitry 1010 and entities and means operatively coupled thereto (such as a communication interface 1020 and storage medium 1030) to perform the methods according to the embodiments described herein. The computer program 1120 and / or the computer program product 1110 can therefore provide means for performing any of the steps disclosed herein.

[0076] exist Figure 10 In the examples, computer program product 1110 is shown as an optical disc, such as a CD (compressed optical disc), DVD (digital versatile optical disc), or Blu-ray disc. Computer program product 1110 can also be implemented as a memory, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM), and more specifically, as a non-volatile storage medium in an external memory device, such as a USB (Universal Serial Bus) memory or flash memory, such as a small flash memory. Therefore, although computer program 1120 is schematically shown herein as a track on the described optical disc, computer program 1120 can be stored in any manner suitable for computer program product 1110.

[0077] The concept of the present invention has been described above primarily with reference to several embodiments. However, as will be readily understood by those skilled in the art, other embodiments besides those disclosed above are also possible within the scope of the concept of the present invention as defined by the appended claims.

Claims

1. A method for detecting reflections of an object (150-1) in an image frame sequence (200, 800, 900), the method being executed by a controller (110), the method comprising: Detect (S102) objects (150-1, 150-2) of a given type in the image frame sequence (200, 800, 900), wherein the given type is a person, an animal or a vehicle, and determine a detection score for each detected object (150-1, 150-2); For each detected object (150-1, 150-2), a distance ratio between unfiltered object location indicators identified for the detected objects (150-1, 150-2) is determined (S104). The unfiltered object location indicators are identified in the current image frame (200, 800, 900) of the image frame sequence (200, 800, 900). The object location indicator represents the point of interest of the target object being tracked from one image frame to the next. The distance ratio of the detected objects (150-1, 150-2) defines the normalized size of the detected objects (150-1, 150-2). The unfiltered object location indicators are determined for all detected objects (150-1, 150-2) without any of the detected objects (150-1, 150-2) being filtered out due to their detection scores being below a detection threshold. For each detected object (150-1, 150-2), determine (S106) a displacement factor between the current position (220-1, 220-2) of the detected object (150-1, 150-2) in the current image frame (200, 800, 900) and the previous position of the detected object (150-1, 150-2) in a previous image frame in the image frame sequence (200, 800, 900). The displacement factor of the detected object (150-1, 150-2) defines the normalized movement of the detected object (150-1, 150-2), wherein the normalized movement refers to movement under a common scaling factor relative to the distance of the detected object (150-1, 150-2) from the camera; and In response to the discovery of a match between the normalized size and the normalized movement of two detected objects (150-1, 150-2) among the detected objects (150-1, 150-2), it is determined (S110) that the detected object with the lower detection score among the two detected objects (150-1, 150-2) is a reflection of the other detected object among the two detected objects (150-1, 150-2).

2. The method according to claim 1, wherein, The method further includes: The match between the normalized size and the normalized movement of two detected objects (150-1, 150-2) is found (S108).

3. The method according to claim 2, wherein, The matching was found to include: Confirm (S108a) that the size difference between the normalized size of one of the two detected objects (150-1, 150-2) and the normalized size of the other detected object (150-1, 150-2) is less than a size threshold; and Confirmation (S108b) that the movement difference between the normalized movement of one of the two detected objects (150-1, 150-2) and the normalized movement of the other detected object is less than a movement threshold.

4. The method according to claim 1, wherein, The method further includes: An action (S112) is performed on the detected object (150-2) that is a reflection of another detected object (150-1), wherein the action involves any of the following: applying a mask (810-2) to the detected object (150-2) that is a reflection of another detected object (150-1) to filter out the detected object (150-2) that is a reflection of another detected object (150-1).

5. The method according to claim 1, wherein, The method further includes: The position (230) of the reflective surface (210) is determined (S114) as the midpoint between the positions (220-1, 220-2) of the two detected objects (150-1, 150-2), wherein the reflective surface (210) is such that one of the detected objects (150-1, 150-2) is a reflection of the other detected object (150-1, 150-2).

6. The method according to claim 5, wherein, The knowledge of the position (230) of the reflective surface (210) is used to detect other objects in the image frame sequence (200, 800, 900).

7. The method according to claim 1, wherein, Each object location indicator in the object location indicator is a common object COCO in the environment, a keypoint (410-1: 417b-1, 410-2: 417b-2), or a MediaPipe pose keypoint.

8. The method according to claim 1, wherein, The object location indicator is the corner point (320s, 320b) and center point (330) of the bounding box (310).

9. A controller (110) for detecting reflections of an object (150-1) in an image frame sequence (200, 800, 900), the controller (110) comprising processing circuitry (210) configured to cause the controller (110) to: Detect objects of a given type (150-1, 150-2) in the image frame sequence (200, 800, 900), wherein, The given type is a person, an animal, or a vehicle, and a detection score is determined for each detected object (150-1, 150-2); For each detected object (150-1, 150-2), a distance ratio is determined between unfiltered object location indicators identified for the detected object (150-1, 150-2), the unfiltered object location indicators being identified in the current image frame 20 (200, 800, 900) of the image frame sequence (200, 800, 900), wherein the object location indicator represents the point of interest of the target object being tracked from one image frame to the next, the distance ratio of the detected objects (150-1, 150-2) defining the normalized size of the detected objects (150-1, 150-2), wherein the unfiltered object location indicators are determined for all detected objects (150-1, 150-2), and no detected object (150-1, 150-2) is filtered out because its detection score is below the detection threshold; For each detected object (150-1, 150-2), a displacement factor is determined between the current position (220-1, 220-2) of the detected object (150-1, 150-2) in the current image frame (200, 800, 900) and the previous position of the detected object (150-1, 150-2) in a previous image frame in the image frame sequence (200, 800, 900). This displacement factor defines the normalized movement of the detected object (150-1, 150-2), where the normalized movement refers to movement under a common scaling factor relative to the distance of the detected object (150-1, 150-2) from the camera; and In response to the discovery of a match between the normalized size and the normalized movement of two detected objects (150-1, 150-2) among the detected objects (150-1, 150-2), it is determined that the detected object with the lower detection score among the two detected objects (150-1, 150-2) is a reflection of the other detected object among the two detected objects (150-1, 150-2).

10. A video surveillance system (100) comprising a controller (110) according to claim 9 and a camera (120) for capturing the image frame sequence (200, 800, 900).

11. A computer program (1120) for detecting reflections of an object (150-1) in an image frame sequence (200, 800, 900), the computer program comprising computer code that, when executed on a processing circuit (210) of a controller (110), causes the controller (110) to: Detect (S102) objects (150-1, 150-2) of a given type in the image frame sequence (200, 800, 900), wherein, The given type is a person, an animal, or a vehicle, and a detection score is determined for each detected object (150-1, 150-2); For each detected object (150-1, 150-2), a distance ratio between unfiltered object location indicators identified for the detected objects (150-1, 150-2) is determined (S104). The unfiltered object location indicators are identified in the current image frame (200, 800, 900) of the image frame sequence (200, 800, 900). The object location indicator represents the point of interest of the target object being tracked from one image frame to the next. The distance ratio of the detected objects (150-1, 150-2) defines the normalized size of the detected objects (150-1, 150-2). The unfiltered object location indicators are determined for all detected objects (150-1, 150-2) without any of the detected objects (150-1, 150-2) being filtered out due to their detection scores being below a detection threshold. For each detected object (150-1, 150-2), determine (S106) a displacement factor between the current position (220-1, 220-2) of the detected object (150-1, 150-2) in the current image frame (200, 800, 900) and the previous position of the detected object (150-1, 150-2) in a previous image frame in the image frame sequence (200, 800, 900). The displacement factor of the detected object (150-1, 150-2) defines the normalized movement of the detected object (150-1, 150-2), wherein the normalized movement refers to movement under a common scaling factor relative to the distance of the detected object (150-1, 150-2) from the camera; and In response to the discovery of a match between the normalized size and the normalized movement of two detected objects (150-1, 150-2) among the detected objects (150-1, 150-2), it is determined (S110) that the detected object with the lower detection score among the two detected objects (150-1, 150-2) is a reflection of the other detected object among the two detected objects (150-1, 150-2).

12. A computer program product (1110) comprising a computer program (1120) according to claim 11 and a computer-readable storage medium (1130) on which the computer program is stored.

Images