Method and system for identifying reflections in thermal images

Abstract

Methods and systems for identifying reflections in thermal images are disclosed. The method comprises detecting a reflection in a first thermal image captured (S1) by a thermal image sensor. In the method, a second image is captured (S2) by a visible light sensor, a near infrared sensor or a short wave infrared sensor, the second image having a field of view overlapping the field of view of the thermal image. A relationship between coordinates in the thermal image and the second image is determined (S3). A first object is detected (S4) in a first location in the thermal image. A candidate image region in the second image in a second location corresponding to the first location is analyzed (S5) to determine whether there is an object equivalent to the first object according to the relationship between coordinates in the thermal image and the second image. In response to (S7) determining that there is no equivalent object in the candidate image region (A c ), it is determined (S8) that the first object is a reflection.

Description

Technical Field

[0001] This invention relates to the field of thermal imaging, and more specifically, to the problem of reflection in thermal images. Background Technology

[0002] Thermal cameras are used in a variety of surveillance applications for security and safety. Because thermal cameras do not require visible light to capture images, they are an attractive option in low-light conditions where cameras using visible light may be less useful. Thermal cameras can also cover large areas where the object of interest has a temperature deviating significantly from the expected temperature of its surroundings. Therefore, thermal cameras are commonly used in security applications such as perimeter surveillance. They can also monitor areas where personal integrity is critical, as thermal cameras can detect intruders without revealing a person's identity. For example, thermal imaging can show someone walking in a school corridor at night without compromising the privacy of students and teachers.

[0003] For safety reasons, thermal cameras are suitable for monitoring industry and infrastructure. For example, they can be used to monitor power generation facilities and substations so that alarms can be triggered when equipment overheats. They can also be used to monitor industrial processes with a risk of spontaneous combustion, as well as to monitor fires, such as in garbage dumps and silos.

[0004] Many thermal camera systems rely on automated monitoring, using motion detection or temperature alarms. Therefore, the images do not need to be continuously viewed by a human observer. However, there are also various thermal camera systems where operators monitor the displayed thermal images in real time. Just like images taken by cameras using visible light, thermal cameras are also affected by reflections. Reflections may arise from reflective surfaces in the monitored scene, or they may be due to reflections from the camera itself. Regardless of the source, such reflections can be offensive or confusing to human observers. They can also lead to false detections in automated analysis algorithms. Therefore, it is necessary to detect reflections in thermal images so that they can be removed from the image or otherwise compensated for.

[0005] Batchuluun et al. proposed a method for detecting reflections in thermal images in their paper "Research on Removing Thermal Reflections" (IEEE Access, Vol. 7, pp. 174597-174611). This proposed method is based on deep learning. Summary of the Invention

[0006] It is an object of the present invention to provide a method of detecting reflections in a thermal image. Another object is to provide a method for reflection detection that makes it possible to locate reflections in a thermal image such that the reflections can be removed or compensated for. A further object is to provide a system and a thermal camera that is capable of detecting reflections in a thermal image. Yet another object is to provide a system and a thermal camera that makes it possible to locate reflections in such a way that they can be removed or otherwise compensated for from the thermal image.

[0007] The invention is defined by the appended claims.

[0008] According to a first aspect, all or at least some of the above objects are achieved by a computer-implemented method of detecting reflections in a first thermal image captured by a thermal image sensor, the method comprising: capturing a second image by a visible light sensor, a near infrared sensor or a short wave infrared sensor, the field of view of the visible light sensor, the near infrared sensor or the short wave infrared sensor overlapping the field of view of the thermal image, determining a mapping from coordinates in the thermal image to coordinates in the second image, detecting a first object in a first location in the thermal image, analyzing a candidate image region in the second image in a second location to determine whether there is an object in the candidate image region that is equivalent to the first object, the second location corresponding to the first location according to the relationship between the coordinates in the thermal image and the coordinates in the second image, and in response to determining that there is no equivalent object in the candidate image region, determining that the first object is a reflection.

[0009] Near infrared radiation is hereinafter abbreviated NIR, and short wave infrared radiation is abbreviated SWIR.

[0010] The method defined in the first aspect of the invention provides a convenient way of detecting reflections in a thermal image, which can be implemented using relatively low-cost components. Adding a visible light camera, a NIR camera or a SWIR camera to a thermal camera is typically much less costly than adding a second thermal camera. Adding a second thermal camera, pointing at the same area in the monitored scene from another angle, can provide another way of detecting reflections compensation.

[0011] The term "equivalent object" denotes herein an object that is sufficiently similar to a first object. It is well known that objects look different in thermal images than in visible light images, NIR images or SWIR images. Thus, the same physical object in a captured scene will not look the same in a thermal image and a second image. However, if the shape and size of the object in the thermal image represents the same physical object in the captured scene, it can be expected that they are similar to the shape and size of the object in the second image. In this case, the object in the second image will be considered equivalent to the first object. It can be clearly understood that if the first object has been detected in the thermal image and no object is found at the corresponding location in the second image, there is no object in the second image that is equivalent to the first object. It can also be understood that if an object is also found at the corresponding location in the second image, but that object is clearly different from the object expected in the thermal image and the object in the second image to represent the same physical object in the scene, the object found in the second image is not an object equivalent to the first object.

[0012] In some variants, analyzing the candidate image region comprises comparing the appearance of the first object to the appearance of the candidate image region to determine whether there is an object in the candidate image region that is equivalent to the first object, and in response to the appearance of the first object differing by more than a first threshold amount, determining that the first object is a reflection. In some images, the appearance of the candidate image region can be such that no object is detected in the candidate image region. In other images, the appearance of the candidate image region can be such that an object is detected there. Such an object can or can not be sufficiently similar to the first object to be determined to be an equivalent object.

[0013] According to a variant of the method, the appearance is at least one from the group consisting of shape, texture, pattern and contrast. Various methods of analyzing these properties of an image are well known in image processing, both for thermal images and for visible light images, NIR images and SWIR images.

[0014] Analyzing the candidate image region can comprise analyzing whether there is an object in the candidate image region to determine whether there is an object in the candidate image region that is equivalent to the first object, and in response to determining that there is no object in the candidate image region, it can be determined that the first object is a reflection. This provides a simple way of determining that the first object is a reflection. If an object detection algorithm is used that is able to distinguish between classes of objects, the method can comprise in response to determining that there is no object in the candidate image region that is of the same class as the first object, determining that the first object is a reflection. This will mean that there is no object at all in the candidate image region, or an object of a different class than the first object.

[0015] According to some variants of the method, another object in the third location in the thermal image is detected, and analyzing the candidate image region comprises detecting a corresponding object in a fourth location in the second image, the fourth location corresponding to the third location according to the relationship between the coordinates in the thermal image and the coordinates in the second image, comparing the appearance of the corresponding object to the appearance of the candidate image region to determine whether the corresponding object is an object equivalent to the first object, the method further comprising, in response to the appearance of the corresponding object differing from the appearance of the candidate image region by more than a second threshold amount, determining that the first object or the other object is a reflection, and in response to the contrast value of the candidate image region differing from the contrast value of the corresponding object by more than a second threshold amount, determining that the other object is a reflection. As used herein, the corresponding object is an object in the second image that corresponds to the other object detected in the thermal image. If the first object and the other object have been detected in the thermal image, one hypothesis that can be formed is that one of these objects represents an actual object in the scene, while the other is a reflection. By looking for a corresponding pair of objects in the second image, i.e. a possible object in the candidate image region and an object that corresponds to the other object, a comparison can be made in the second image. If no object is present in the candidate image region, only the other object has a correspondence in the second image, and it can be inferred that the first object is a reflection. If an object is also found in the candidate image region, a comparison can be made between the two objects found in the second image. If they differ significantly from each other, it can be concluded that they do not represent two equivalent objects in the scene. From this, it can be inferred that one of the objects detected in the thermal image is a reflection. Studying the contrast of the candidate image region and the corresponding object provides a convenient way to make the comparison.

[0016] According to variants of the method, the appearance is a contrast. In this variant, the method comprises, in response to the contrast value of the corresponding object differing from the contrast value of the candidate image region by more than a second threshold amount, determining that the other object is a reflection, and in response to the contrast value of the candidate image region differing from the contrast value of the corresponding object by more than a second threshold amount, determining that the other object is a reflection.

[0017] The method can further comprise determining that the other object is a source object, the first object being a reflection of the source object. Information about the source object and its appearance can help compensate for its reflection.

[0018] In some variants, the method further comprises, in response to determining that the first object is a reflection, compensating for the reflection in the thermal image. Thereby, a better viewing experience can be provided for the operator. This in turn can reduce the risk of the operator missing important events in the thermography. Compensating for the reflection can also reduce the risk of false detections in the automatic analysis of the thermal image.

[0019] The method can further include detecting another object in the thermal image and determining that the other object is the source object and the first object is a reflection of the source object. Information about the source object and its appearance can help compensate for its reflection.

[0020] Compensating for the reflection in the thermal image can include determining a reflection intensity gradient between a first pair of reference points of the first object, determining a source intensity gradient between a corresponding second pair of reference points of the source object, for each pixel within the first object, calculating a gradient ratio between the reflection gradient and the source gradient, and calculating a compensated intensity value by calculating a difference between a captured intensity value of the pixel and a captured intensity value of a corresponding pixel of the source object multiplied by the gradient ratio. This method can more or less remove the reflection from the thermal image, thereby displaying a thermal image that is closer to the truth.

[0021] In other variants, compensating for the reflection in the thermal image includes interpolating between intensity values of pixels around the first object. This is a simple way of compensating for the reflection and does not require finding the source object. The result can not necessarily represent the true conditions in the captured scene, but it can be good enough given that it provides the observer with a thermal image that is free of objectionable reflections.

[0022] In other variants, compensating for the reflection in the thermal image includes storing background intensity values captured when there is no reflection in the location of the first object, and for each pixel within the first object, replacing the captured intensity value of the pixel with the stored pixel background intensity value. Like the interpolation method, this background data method can provide good enough compensation in a simple way.

[0023] In other variants, compensating for the reflection in the thermal image includes, for each pixel within the first object, selecting a representative pixel outside the location of the first object and replacing the captured intensity value of the pixel with the captured intensity value of the representative pixel. In this way, the reflection can be compensated for by patching with thermal image data from a suitable image area outside the reflection. This can also be a simple way of providing good enough compensation.

[0024] According to a second aspect, the above-mentioned objects are all or at least partly achieved by a system for detecting reflections in a first thermal image captured by a thermal image sensor, the system comprising: an image receiver configured to receive a thermal image and a second image captured by a visible light image sensor, a near-infrared sensor or a short-wave infrared sensor, the thermal image and the second image having overlapping fields of view. The system further comprises: a mapping module configured to determine a mapping from coordinates in the thermal image to coordinates in the second image; an object detector configured to detect a first object in a first location in the thermal image; an image region analyzer configured to analyze a candidate image region in the second image in a second location corresponding to the first location according to the relationship between the coordinates in the thermal image and the coordinates in the visible second image, to determine whether an object equivalent to the first object is present in the candidate image region. The system further comprises a reflection determination module configured to determine that the first object is a reflection in response to determining that no equivalent object is present in the candidate image region. The system makes it possible to detect thermal reflections in a relatively simple manner. Successively detecting reflections in thermal images makes it possible to remove reflections from thermal images, making thermal images more usable for surveillance purposes.

[0025] The system can comprise a thermal camera and a visible light camera, a near-infrared camera or a short-wave infrared camera. Supplementing a thermal camera with a visible light camera or a near-infrared camera is a relatively inexpensive way that provides the means needed to detect reflections in thermal images captured by the thermal camera. Although typically more expensive than visible light cameras and near-infrared cameras, adding a short-wave infrared camera can be a less expensive option than, for example, adding another long-wave infrared camera.

[0026] The system of the second aspect can typically be implemented in the same way and have the same advantages as the method of the first aspect.

[0027] According to a third aspect, the above-mentioned objects are all or at least partly achieved by a thermal camera comprising a thermal image sensor and a system according to the second aspect. The thermal camera of the third aspect can typically be implemented in the same way and have the same advantages as the method of the first aspect and the system of the second aspect.

[0028] According to a fourth aspect, the above-mentioned objects are all or at least partly achieved by a non-transitory computer-readable storage medium having stored thereon instructions for implementing the method according to the first aspect when executed on a device having processing capability. The fourth aspect can vary in the same way and have the same advantages as the method of the first aspect.

[0029] A “thermal image” in this application refers to an image that captures long-wave infrared radiation or mid-wave infrared radiation. In the following, they will be referred to by their abbreviations “LWIR” and “MWIR”, respectively.

[0030] Further scope of the applicability of the present application will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the application, are given by way of illustration only, since various changes and modifications within the scope of the application will become apparent to those skilled in the art from this detailed description.

[0031] It should be understood, therefore, that the present application is not limited to the particular combinations of parts and steps described or the particular steps described, as such parts and steps can vary. It should be further understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting, as the scope of the present application will be limited only by the appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0032] The present application will now be described in more detail, by way of example, and with reference to the accompanying schematic drawings, in which:

[0033] Figure 1 is an illustration of a thermal image,

[0034] Figure 2 is an illustration of a visible light image of the same scene as the thermal image in Figure 1

[0035] Figure 3 is a view of the scene monitored by the first thermal camera and the second visible light camera,

[0036] Figure 4 shows an object in the form of a face detected in the thermal image of Figure 1

[0037] Figure 5 shows the area marked in the visible light image of Figure 2 corresponding to the face marked in the thermal image of Figure 4

[0038] Figure 6 shows an interpolation method for compensating for reflections in the thermal image of Figure 1

[0039] Figure 7 shows a background image of the scene captured in the thermal image of Figure 1

[0040] Figure 8 Figure 1 ​​​​​​Cloning method of the reflection in the thermal image,

[0041] Figure 9 A face detected in the center of the image in Figure 4 is shown, where a Sobel filter is used to analyze the gradients,

[0042] Figure 10 A face detected on the right side of the image in Figure 4 is shown, where a Sobel filter is applied,

[0043] Figure 11 A difference image showing the difference between the gradients in Figure 9 and the gradients in Figure 10 is shown,

[0044] Figure 12 is a version of the thermal image in Figure 1 where the reflective surface on the right side has been removed,

[0045] Figure 13 is a simplified illustration of a thermal image with a source object and a reflection,

[0046] Figure 14 is an illustration of another thermal image,

[0047] Figure 15 A visible light image showing the same scene as the thermal image in Figure 14 is shown,

[0048] Figure 16 is a flowchart illustrating a variant of the method of the invention,

[0049] Figure 17 is a block diagram of a system for detecting a reflection in a thermal image, and

[0050] Figure 18 is a block diagram of a thermal camera. DETAILED DESCRIPTION

[0051] A first image 1 is shown in Figure 1 . The first image 1 is a thermal image captured using a thermal image sensor, such as a microbolometer. In this example, the thermal image captures LWIR. As mentioned above, it can also capture MWIR. In Figure 2A second image 2 is shown in the middle. In this example, the second image 2 has been captured by a visible light sensor and will be referred to as a visible light image. The thermal image 1 and the visible light image 2 capture the same scene, in which a person 3 can be seen walking in a corridor 4. At the left and right side, the corridor 4 is lined by glass surfaces 5, 6. The thermal image 1 and the visible light image 2 do not need to have exactly the same field of view, but their fields of view must have an overlap. More specifically, they need to overlap in the area in which reflections in the thermal image 1 should be detected. For illustrative purposes, in the shown example, the fields of view of the thermal image 1 and the visible light image 2 are substantially the same.

[0052] From Figure 1 the thermal image 1 and Figure 2 the visible light image 2, it can be seen that the person 3 is reflected in the glass surfaces 5, 6. This reflection can be objectionable to an operator viewing the images 1, 2. Furthermore, if automated event detection using image analysis is used, the reflection can cause false alarms. Therefore, it is of interest to remove the reflection from the images. In many cases, it is more difficult for a human observer viewing a thermal image or an automated analysis algorithm analyzing a thermal image to distinguish what is a real object and what is a reflection compared to viewing a visible light image. Specular reflection of LWIR causes this problem, as it makes the reflection very similar to the real object causing the reflection. Therefore, it can be particularly important to remove the reflection from the thermal image. It can also be noted that it can be relatively easy for a human observer to distinguish reflections from real objects in a scene such as shown in Figure 1 the visible light image 2, it can be seen that the person 3 is reflected in the glass surfaces 5, 6. This reflection can be objectionable to an operator viewing the images 1, 2. Furthermore, if automated event detection using image analysis is used, the reflection can cause false alarms. Therefore, it is of interest to remove the reflection from the images. In many cases, it is more difficult for a human observer viewing a thermal image or an automated analysis algorithm analyzing a thermal image to distinguish what is a real object and what is a reflection compared to viewing a visible light image. Specular reflection of LWIR causes this problem, as it makes the reflection very similar to the real object causing the reflection. Therefore, it can be particularly important to remove the reflection from the thermal image. It can also be noted that it can be relatively easy for a human observer to distinguish reflections from real objects in a scene such as shown in

[0053] In this case, it can be noted that reflections can occur in a surveillance scene, such as on glass surfaces 5, 6, or inside the thermal camera, such as inside the view window, on the lens surface, and on the sensor cover glass.

[0054] The glass surface 5, 6 has different reflective properties for LWIR and visible light. The reason for this difference is that the LWIR sensor captures the self-emitted radiation of objects in the scene, while the visible light sensor mainly captures light reflected by objects in the scene. In some cases, the reflection of visible light can be more diffusely reflected, while the reflection of LWIR can be more specularly reflected. Thus, while the person 3 can easily be identified as the same object in the thermal image 1 and the visible light image 2, the reflection of the person 3 differs more between the thermal image 1 and the visible light image 2. The inventors of the present invention have realized that this difference can be used to advantageously detect reflections in the thermal image. By detecting objects in the thermal image and analyzing the corresponding areas in the visible light image, it is possible to find objects in the thermal image that lack a counterpart in the visible light image or have a different appearance in the visible light image. By this method, reflections in the thermal image can be detected, which will be explained in further detail below. In some variants of the method, both the real object and its reflection need to appear in the thermal image in order to be able to determine that a reflection is present in the thermal image. In other variants, it is sufficient that only the reflection appears in the thermal image, while the real object can be outside the field of view of the thermal camera.

[0055] Figure 3 A scene monitored by the thermal camera 7 and the visible light camera 8 is shown.

[0056] According to a variant of the method of the present invention, a first image 1 is captured. This first image is a thermal image 1 captured by the thermal camera 7, which also has a thermal sensor. A second image 2 is also captured. This second image is a visible light image 2, a NIR image or a SWIR image. In the example discussed below, the second image is a visible light image 2 captured by the visible light camera 8, which has a visible light sensor. The skilled person will understand that the second image can also be a NIR image or a SWIR image. The cameras and sensors will be discussed further later.

[0057] As mentioned above, the fields of view of the thermal image 1 and the visible light image 2 have an overlap. If the fields of view coincide, it is possible to detect reflections in the entire thermal image. If only a part of the field of view of the thermal image 1 is overlapped by the field of view of the visible light image 2, then using the method of the present invention will likely only detect reflections in the overlapping area.

[0058] The coordinates in the thermal image 1 are mapped to coordinates in the visible light image 2. This can be done in any suitable way, and several known methods can be used by the person skilled in the art. For example, the mapping function can be calculated from manual input, for example by an installer mounting the thermal camera and the visible light camera. The installer can identify features in the thermal image captured by the thermal camera and indicate them, for example by clicking on them in the thermal image using a computer mouse. The installer can then identify corresponding features in the visible light image captured by the visible light camera and indicate them in the same way as in the thermal image. When a sufficient number of feature pairs have been indicated, a mapping function describing the relationship between the coordinates in the thermal image and the coordinates in the visible light image can be calculated for the entire image or at least for the overlapping area. Automatic mapping methods are also known. Some automatic mapping methods are based on finding a plurality of features, such as corners or edges of objects, that appear in both images and calculating the relationship between their coordinates. Other automatic mapping methods employ a route via real-world coordinates, calculating a homography for each camera based on the respective camera matrix. If the fields of view of the two sensors coincide, it can be assumed that the coordinates in one image are identical to the coordinates in the other image. Examples of automatic mapping methods are described in Blum et al., 2006, "Multisensor Image Fusion and Its Applications", for example see chapters 1 and 3, and Yang et al., 2011, "Image Fusion and Its Applications", "Automatic optical and infrared image registration for plant water stress sensing". The coordinate mapping can advantageously be performed once when the cameras are installed, and then the established relationship can be retrieved when reflection detection is needed. If a dual-sensor camera is used, instead of two separate cameras, the mapping can have been performed already at the time of camera manufacture. If two separate cameras are used, and the device is displaced due to vibration or is intentionally changed, an updated mapping can be made to determine a new relationship between the coordinates in the thermal image and the coordinates in the visible light image. Such an update can be scheduled to be performed periodically, or it can be performed temporarily.

[0059] A first object 3 is detected at a first position (x1, y1 ) in the thermal image 1. In Figure 4 The first object 9 is a reflective surface in the example shown. In Figure 4 The first object 9 is marked by a bounding box. With the knowledge established by the above-described mapping of the relationship between the coordinates in the thermal image 1 and the visible light image 2, a candidate image region A c The second position (x2, y2) corresponds to the first position (x1, y1 ) in the thermal image 1. Assuming that the mapping has been done correctly, the first position (x1, y1 ) in the thermal image 1 and the second position (x2, y2) in the visible light image both represent the same real-world position in the monitored scene.

[0060] Analyzing the candidate image region A c to determine whether an object equivalent to the first object 9 is present in the visible light image 2. As mentioned before, this analysis can be performed in different ways, depending on whether only a potential reflection is present in the thermal image, or whether a real object and a reflection can be detected in the thermal image. The appearance of the candidate image region A c is compared to the appearance of the first object 9. This analysis can result in the object being found in the visible light image 2 as well, or the object not being found in the visible light image. If the object is not found in the candidate image region in the visible light image 2, it is easily understood that the appearance of the first object is clearly different from the appearance of the candidate image region A c . It is also understood that no object equivalent to the first object 9 is present in the visible light image. If the object is found in the candidate image region of the visible light image as well, this object can be referred to as a second object. However, since the present invention is applicable both in the case that no object is found in the candidate image region A c , and in the case that an object is found in the candidate image region A c , in most of the following text, no reference will be made to the second object, but to the candidate image region A c . Even if a second object is found in the candidate image region A c , the appearance of the first object 9 can be different or can not be different from the appearance of the candidate image region A c . If the second object is clearly different from the first object, it is not an equivalent object. As mentioned above, if an object detection algorithm is used that determines an object class (e.g. person or vehicle) for each detected object, it can be determined that the second object is not an equivalent object if the object class of the first object is different from the object class of the second object.

[0061] The appearance of the first object 1 and the candidate image region A c can be a shape, a texture or a pattern of the corresponding object or region. A combination of two or more of a shape, a texture and a pattern can also be used as an appearance. Given that the thermal image and the visible light image are intrinsically different, the appearance of the first object 9 and the candidate image region A c need not be identical, as both are considered to be sufficiently similar to represent the same real-world object in the monitored scene. A first threshold amount d1 is determined, and this first threshold amount d1 is used when comparing the appearance of the first object 9 and the candidate image region A c . If the appearance of the first object 9 differs less than the first threshold amount d1 from the appearance of the candidate image region A c , the first object 9 and the candidate image region A cSimilar. This similarity indicates that the same real-world object is represented in thermal image 1 and visible light image 2, and thus the first object 9 is real. This can also be referred to as the first object 9 having a candidate image region A c in the visible light image c On the other hand, if the first object 1 and the appearance of the candidate image region A c differ by more than a threshold amount, they are not considered similar, and this indicates that the first object 9 is not a real object. In other words, there is no object in the candidate image region A c that is equivalent to the first object 9. Thus, if the appearance of the first object 9 and the appearance of the candidate image region A c differ by more than a threshold amount δ, the first object 9 is determined to be a reflection. The first threshold amount δ1 can be established empirically, and will typically depend on the material causing the reflection. For example, if Michelson contrast is used as a measure of appearance, it can be reasonable to assume that if the contrast value of the candidate image region is less than 10% of the contrast value of the first object, the first object is a reflection.

[0062] If both the reflection and the real object are within the field of view of the thermal camera 7, the analysis of the candidate image region A c can be done in a different way. In this case, the purpose of the analysis is also to determine whether there is an object in the candidate image region A c that is equivalent to the first object in the thermal image. In addition to the first object 1, another object is detected in the thermal image 1, also referred to as a third object 14, at a third position (x3, y3). If the first object 9 and the third object are sufficiently similar, it can be assumed that one of them is a real object and the other is a reflection of the real object. To determine whether either of the first object 9 and the third object 14 is a reflection, a potential pair of objects is searched for in the visible light image 2. As mentioned previously, the relationship between the coordinates in the thermal image and the coordinates in the visible light image is known. Thus, a fourth position (x4, y4) in the visible light image that corresponds to the third position (x3, y3) can be found. If a fourth object is detected at the fourth position, this fourth object is the object that corresponds to the third object 14. Thus, the fourth object can also be referred to as the corresponding object. If no object is found at the fourth position, it can be determined in the same way as in the case described above where only one object is detected in the thermal image that the third object is a reflection. If a corresponding object is found at the fourth position, a comparison is made between the candidate image region A c and the fourth object. A convenient way to compare the candidate image region A c and the fourth object is to study the contrast values. For example, Michelson contrast can be calculated for the candidate image region and the fourth object. The candidate image region A cThe contrast value of the fourth object is compared to the contrast value of the candidate image region. If the contrast value of the fourth object exceeds the contrast value of the candidate image region by more than a second threshold amount δ2, it can be assumed that the fourth object is a real object and that the candidate image region contains no object or contains a reflection of the fourth object. Therefore, it can be determined that the first object in the thermal image is a reflection. Conversely, if the contrast value of the candidate image region exceeds the contrast value of the fourth object by more than the second threshold amount δ2, it can be assumed that there is a real object in the candidate image region and that the fourth object is a reflection. Therefore, it is determined that the third object is a reflection. The second threshold amount δ2 can be established according to the same principles as discussed above for the first threshold amount δ1.

[0063] Once it is determined that the first object 9 is a reflection, different actions can be taken. In the following, the first object 9 will be discussed, but it will be understood that the same actions can be taken for the third object if it is determined to be a reflection. For example, if an automatic event detection algorithm is applied to the image, the detection of the first object 9 can be suppressed so that it does not trigger any events that should only be triggered by real objects. If the thermal image 1 is to be observed by a human observer, a flag or other indication can be added to the image indicating that the first object 9 is a reflection and should be ignored. However, it can be more useful to remove the first object 9 from the thermal image 1 so that the human observer is not distracted by the reflection. For an automatic analysis algorithm, it can be less important if the reflection is removed from the thermal image or if the algorithm is instructed to ignore the reflection in another way.

[0064] If it has been determined that the first object 9 is a reflection, the area of the first position (x1, y1 ) where the first object 9 was detected can be marked and stored as a reflection area. This information can then be used for subsequently captured thermal images so that if a subsequent object is detected in this reflection area, it can be determined that the subsequent object is also a reflection without having to be compared to a corresponding visible light image. This can make the reflection detection more efficient as it requires less computation.

[0065] Although effective, the assumption that any object detected in a region that has been identified as a reflection region is a reflection carries the risk of false positives, i.e. real objects being wrongly identified as reflections. For example, sometimes real objects can appear in front of a reflection region. To significantly reduce this risk, preferably, each time an object is detected in the thermal image, a comparison is made with a candidate image region in the corresponding visible light image. By taking into account factors such as the angle of view of the thermal camera and the lighting conditions in the scene, the risk of false positives can also be reduced. The reflection characteristics of surfaces in the monitored scene can differ with different angles of view and different lighting conditions. Therefore, a compromise between efficiency and the risk of false reflection detection can be that any object detected in a region that has been identified as a reflection region is assumed to be a reflection only if the angle of view of the thermal camera is the same as the angle of view at which the reflection region was identified and / or only if the same lighting conditions prevail. Using this approach, if the thermal camera has pan and tilt capabilities, reflection regions can be identified for a plurality of different pan / tilt positions, such as positions on a so-called patrol route, and any object detected in a reflection region identified for a given position is considered to be a reflection when the thermal camera 7 is pointed at that position. The identification of reflection regions can be updated at regular intervals or on an ad hoc basis.

[0066] Whether the first object is determined to be a reflection by actual comparison with the visible light image or by detection in a region that has been identified as a reflection region, the advantage of having determined that it is a reflection is that it is possible to compensate for the reflection in the thermal image.

[0067] There are several possible ways of compensation. As mentioned above, for an automatic analysis algorithm, it is not necessary to remove the reflection from the thermal image. Rather, a better solution can be to instruct the algorithm to ignore the first object. For a human observer, it is often preferable to remove the reflection from the thermal image, or at least to make the reflection less conspicuous. In some cases, this can be sufficient to make the reflection less conspicuous so that it does not distract the observer. In some cases, it can be necessary to remove the reflection from the thermal image.

[0068] One way of removing or compensating for the reflection is to use interpolation to fill in the missing data in the thermal image. For example, the missing data can be filled in by interpolating between the data of the thermal image in the regions that are not affected by the reflection. Another way of removing or compensating for the reflection is to use extrapolation to fill in the missing data in the thermal image. For example, the missing data can be filled in by extrapolating from the data of the thermal image in the regions that are not affected by the reflection. Figure 6An explanation is given. The interpolation method is based on the assumption that the area where the reflection occurs is similar to its surroundings. The surrounding pixels 10 in the image area outside the first object 9 are identified and for each pixel inside the area of the first object 9, an interpolated intensity value is calculated based on the intensity values of the surrounding pixels. The interpolation can be more or less complex. In a simple form, the interpolated intensity values can be calculated row by row in the area of the first object 9 based on the intensity values of the surrounding pixels 10 just to the left and right of the row. In a more complex interpolation, more surrounding pixels can be considered for each pixel inside the first object 9. By replacing the intensity values inside the first object 9 with the interpolated intensity values, the reflection can be removed from the thermal image 1. The interpolation provides a computationally simple way to remove the reflection, although it does not necessarily provide a true image of the scene without the reflection. Typically, removing the reflection and replacing it with an interpolation is a practical solution that makes the thermal image 1 more usable for a human observer and less likely to trigger false positives in automatic analysis algorithms. The advantage of low computational cost must be weighed against the risk of removing actual objects that are hidden in the reflection. Therefore, in some cases, a more computationally demanding compensation method can be desirable.

[0069] Another way to remove or compensate for the reflection is to use historical data. This will be discussed with reference to Figure 7 If the reflection does not always occur in the thermal images of the scene, the intensity values of all pixels in the thermal images without the reflection can be stored as a background image 11. At a later point in time, when it has been determined that the first object 9 is a reflection, the reflection can be removed by replacing the intensity values of the pixels in the thermal image 1 that make up the first object 9 with the stored background intensity values of those pixels 12. If the reflection area has been identified, it is not necessary to store the background intensity values of all pixels of the background image 11, but only the background intensity values of the reflection area, for example the pixels marked by 12 in Fig. 5. Figure 7 Similar to the interpolation method, using a background image is a computationally efficient way of compensating for the reflection. However, if real objects occur in the image area of the reflection, using historical data is also accompanied by the risk of missing events in the scene.

[0070] Another way to remove or compensate for the reflection is to patch the area of the first object 9 by cloning another area of the thermal image 1. This can be discussed with reference to Figure 8An image region 13 outside the first object 9 is selected. If there are no reflections, the selected region 13 should preferably represent the image region where the first object 9 appears. In a relatively uniform or calm scene, it can be reasonable to assume that the image region next to the first object 9 is similar to the first object's region without reflections. The image region 13 to be cloned can be selected manually or automatically. If done manually, the selection of the image region 13 to be cloned can advantageously be done at one occasion, for example when the thermal camera 7 is installed. According to this method, the reflections can be removed by replacing the intensity values within the region of the first object 9 with intensity values of corresponding pixels in the selected region 13. This method is also relatively simple to calculate, but has the same caveats as the interpolation method and the background image method described above.

[0071] If the reflections are caused by surfaces in the scene, and not by surfaces inside the thermal camera 7, the reflections can be removed or compensated by studying the source object in the thermal image 1. The source object is the first object 9 as its reflection. It will be referred to as the source object 14. Figure 4 and Figure 9 This method is discussed in more detail.

[0072] As mentioned above, Figure 4 the first object 9 in the thermal image 1 has been determined to be a reflection. Another object, also referred to as a third object 14, is detected at a third position (x3, y3) in the thermal image 1. It is determined that the third object 14 is the source object of which the first object 9 is a reflection. By comparing the third object with the first object 9, the third object 14 can be identified as the source object. If the third object 14 and the first object have a similar appearance, the third object 14 can be the source of the first object 9. If the region of the visible light image 2 corresponding to the position of the third object in the thermal image 1 has an appearance similar to the appearance of the third object, the third object 14 can be determined to be a real object and can be determined to be the source object 14 causing the reflections. After the source object 14 has been identified, in which the first object 9 is a reflection, the reflections can be removed or compensated by analyzing the gradient of the intensity values of the first object 9 and the source object 14. The gradient can be derived, for example, by applying a Sobel filter to the thermal image 1. Alternatively, another high-pass filter can be applied.

[0073] In Figure 9 the first object 1 is shown with a Sobel filter applied. Correspondingly, in Figure 10 the source object 14 is shown with a Sobel filter applied. Figure 9 and Figure 10 The Sobel filtered image in

[0074] determining a reflection intensity gradient G between a first pair of reference points of the first object 9 r . Similarly, a source intensity gradient G s between a corresponding second pair of reference points of the source object 14 is determined r . The gradient ratio R s is calculated by dividing the reflection gradient G G by the source intensity gradient G r .

[0075] For each pixel within the first object 9, a gradient difference Δ G between the reflection intensity gradient G r and the source intensity gradient G s is calculated G . The gradient ratio R G is multiplied by the gradient difference Δ G to calculate a compensated intensity value I comp for the pixel. As Figure 12 illustrated, by replacing the captured intensity value of each pixel within the first object 9 with its compensated intensity value, the reflection can be removed from the thermal image. This gradient method is a way to derive the relative reflectivity of the reflective region, which can then be compensated for.

[0076] With reference Figure 13 to FIG. 6, a simplified example of the gradient method can be schematically described by the following equations:

[0077] G r = I b1 - I a1

[0078] G s = I b2 - I a2

[0079]

[0080] Here, I a1 is the intensity in point P a1 , and I b1 is the intensity in point P b1 in the first object, I a2 is the intensity in point P a2 , and I b2 is the intensity in point P b2 in the source object.

[0081] While the gradient method described here can produce compensated images that more closely resemble realistic images of non-reflective scenes, some considerations remain. If no real object exists in the reflective image region, the compensated image may be close to reality. However, if a real object is present in addition to the reflection, for example because the real object occludes part of the reflective region, compensation using gradients from the source object may distort the intensity values ​​in the image region of the first object 9. Similarly, the importance of not losing the real object needs to be weighed against the importance of avoiding false alarms and unpleasant reflections.

[0082] Figure 14 and Figure 15 It shows the relationship with Figure 1 and Figure 2 Similar image pairs. (And related to...) Figure 1 and Figure 2 The same way of describing Figure 14 A thermal image 21 of the scene is shown, and Figure 15 A visible light image 22 of the same scene is shown. Figure 12 Thermal images 21 and Figure 1 The only significant difference between the thermal images 1 and 21 is that the first object 29 is a reflection caused by a reflective surface inside the thermal camera that captured the thermal image 21, while Figure 1 The first object 9 in the thermal image is caused by reflective surfaces in the scene. Reflections may occur inside the thermal camera due to reflective surfaces on the front glass, lens, and sensor components. The method for detecting reflections caused by reflective surfaces inside the camera is combined with the above. Figure 1 and Figure 2 The method described is the same. Therefore, when the first object 29 has been detected in thermal image 21, in Figure 15 Identify the corresponding candidate image region A in the visible light image 22. c2 And compare the first object 29 with the candidate image region A c2 The appearance. If they differ by more than a threshold amount δ, then the first object 29 is determined to be a reflection.

[0083] Reflections in thermal image 21 can be removed or compensated for in the same manner as described above. Interpolation, cloning, and background image methods can be used without modification, regardless of the cause of the reflection. The gradient method may need modification because identifying the source object can be more complex. Internal reflections will make the first object appear to be at a different distance from the camera compared to the corresponding real object, potentially causing the first object to be out of focus. A modification to the gradient method could be to include the ratio between the average intensity of the source object and the average intensity of the reflection in the calculation, and then replace the pixel intensity in the same manner as described above.

[0084] Now refer to Figure 16 The flowchart in the document outlines the method of the present invention.

[0085] In step S1, a first image is captured. The first image is a thermal image captured by a thermal image sensor. In step S2, a second image is captured by a second sensor. The second sensor is a visible light sensor, a NIR sensor or a SWIR sensor.

[0086] In step S3, a relationship between coordinates in the thermal image and coordinates in the second image is determined. This can be done at any point in time before a further step of the method requires the relationship.

[0087] In step S4, a first object is detected at a first location in the thermal image. A corresponding second location is found in the second image. To find the second location, the relationship between coordinates in the thermal image and coordinates in the second image needs to be known. In step S5, a candidate image region in the second image at the second location is identified.

[0088] In step S6, the candidate image region is analyzed to determine whether an object equivalent to the first object is present in the candidate image region. In step S7, it is checked whether an equivalent object has been found. If it is found that no equivalent object is present in the candidate image region, it is determined in step S8 that the first object is a reflection.

[0089] As mentioned above, the analysis of the candidate image region can be done in different ways. If only the first object is detected in the thermal image and no further or third object is detected, it can be checked whether an object is present in the candidate image region. If no object is found in the candidate image region, it can be concluded that the first object is a reflection. If an object is found in the candidate image region, the appearance of the object can be compared to the appearance of the first object and if they differ sufficiently, it can again be determined that the first object is a reflection.

[0090] Depending on the importance of detecting a possible reflection in the thermal image, the result that the appearance of the first object does not differ from the appearance of the candidate image region by more than a first threshold amount can lead to different conclusions. The same applies if the contrast value of the candidate image region does not differ from the contrast value of the fourth object by more than a second threshold amount. In the simplest solution, if it is found that the difference does not exceed the respective threshold amount, it can be determined that the first object is not a reflection. If it is important not to miss any reflections in the thermal image, finding that the difference does not exceed the threshold amount can lead to the decision to make further analysis. For example, an additional threshold amount can be used which is lower than the first-mentioned threshold amount (i.e. the first threshold amount or the second threshold amount, respectively). If the difference also does not exceed the additional threshold amount, it can be determined that the first object is not a reflection, and if the difference does not exceed the first-mentioned threshold amount but exceeds the additional threshold amount, it can be decided that further analysis is needed.

[0091] ReferenceFigure 17 A system 20 according to embodiments of the application will now be described. The reflection detection system can be used according to the above described method. The system 20 comprises an image receiver 21. The image receiver 21 is configured to receive a thermal image and a second image. As described above, the second image is a visible light image, a NIR image or a SWIR image. The thermal image and the second image have overlapping fields of view.

[0092] The system 20 further comprises a mapping module 22 configured to determine a relationship between coordinates in the thermal image and coordinates in the second image. The relationship can be established by the system itself or can be retrieved from an external device. For example, the relationship can be retrieved in the form of a look-up table or a mathematical formula.

[0093] Furthermore, the system 20 comprises an object detector 23 configured to detect a first object in a first location in the thermal image. The system further comprises an image region analyzer 24 configured to analyze a candidate image region in a second location in the second image to determine whether an object equivalent to the first object is present in the candidate image region. Using the relationship between coordinates in the thermal image and coordinates in the second image, the second location is chosen such that it corresponds to the first location in the thermal image. In other words, both the first location in the thermal image and the second location in the second image should represent the same location in the captured scene.

[0094] Furthermore, in some embodiments, the system 20 comprises a comparison module 25. Depending on the scene, i.e. if only the first object is detected in the thermal image, or if another object is also detected, the comparison module can be configured to compare an appearance of the first object and an appearance of the candidate image region, or to compare a contrast value of the candidate image region and a contrast value of the fourth object. The system 20 further comprises a reflection determination module 26 configured to determine that the first object is a reflection if the result of the analysis of the candidate image region is that no equivalent object is present in the candidate image region.

[0095] The system 20 can further comprise a compensator 27 configured to remove or compensate for detected reflections.

[0096] The system 20 can be incorporated in a thermal camera, such as Figure 3 the thermal camera 7 shown. Figure 18A block diagram of the camera 7 is shown in Fig. 1. The thermal camera 7 has a lens 30 through which a sensor 31 captures the LWIR in a scene. The sensor 31 can be a microbolometer. The thermal camera 7 further comprises an image processor 32, an encoder 33 and a network interface 34 through which images captured by the thermal camera 7 can be transmitted for viewing and / or storage. The thermal camera 7 can further comprise other components, but as these are not essential for the explanation of the invention, they will not be discussed herein. As already mentioned, the thermal camera 7 can comprise the reflection detection system 20, enabling detection of reflections in images captured by the thermal camera 7. If the reflection detection system is incorporated in the thermal camera 7, reflections can be detected and possibly already compensated for before the image is transmitted from the thermal camera 7.

[0097] Instead of incorporating the reflection detection system 20 in the camera 7, the system 20 can be arranged separately and operatively connected to the thermal camera 7. The separate reflection detection system 20 does not need to be directly connected to the thermal camera, but can be connected to or incorporated in a video management system to which images from the thermal camera are transmitted. In this case, one reflection detection system 20 can be used to detect reflections in thermal images from more than one thermal camera 7.

[0098] Whether the reflection detection system is incorporated in the thermal camera or arranged separately, it can further comprise a visible light camera 8, a NIR camera or a SWIR camera.

[0099] The reflection detection system 20 can be implemented by means of hardware, firmware, software or any combination thereof. When implemented as software, the reflection detection system can be provided in the form of computer code or instructions that, when executed on a device having processing capability, will implement the temperature control method described above. Such a device can for example be or can comprise a central processing unit (CPU), a graphics processing unit (GPU), a custom processing device implemented in an integrated circuit, ASIC, FPGA or logic circuit comprising discrete components. When implemented as hardware, the system can comprise circuitry in the form of an arrangement or system of circuits. For example, it can be provided on a chip and can further comprise or otherwise be arranged together with software for performing the processing.

[0100] It will be appreciated that the above-described embodiments can be modified in various ways by a person skilled in the art and still exploit the advantages of the invention as shown by the above-described embodiments. As an example, as mentioned above, the second image does not necessarily have to be a visible light image, but can also be a NIR image or a SWIR image.

[0101] The cameras 7, 8 can be digital cameras, but can also be analog cameras connected to a digitizing device.

[0102] The thermal image sensor and the visible light sensor (or NIR or SWIR sensor) can be mounted in separate cameras, as shown in Figure 3 Alternatively, the dual sensor can be arranged in the same camera. Such dual sensor cameras are known and available from e.g. Axis Communications AB (see e.g. AXIS Q8752-E Dual-Band PTZ Camera).

[0103] Both cameras can have a fixed field of view. Alternatively, one or both can have a variable field of view, with zoom functionality or PTZ functionality (i.e. pan, tilt, zoom functionality).

[0104] The thermal image sensor can be a microbolometer. Alternatively, the thermal image sensor can be any other type of thermal sensor, such as a cooled thermal sensor.

[0105] In the above examples, the contrast value is described as a Michelson contrast. Other contrast measures can be used instead, such as Weber contrast or RMS contrast. The contrast value can also be based on a histogram of the image, studying the difference in intensity and chrominance.

[0106] The second image sensor can be a CCD sensor or a CMOS sensor.

[0107] The application should therefore not be limited to the shown embodiments, but should only be defined by the appended claims.

Claims

1. A computer-implemented method for detecting reflections in a first thermal image (1) captured (S1) by a thermal image sensor (31), the method comprising: A second image (2) is captured (S2) by a visible light sensor, a near-infrared sensor or a short-wave infrared sensor, wherein the visible light sensor, the near-infrared sensor or the short-wave infrared sensor has a field of view that overlaps with the field of view of the thermal image (1); Determine (S3) the mapping from coordinates in the thermal image to coordinates in the second image; Detect (S4) the first object (9) at the first position (x1, y1) in the thermal image (1); Analysis (S5) of the candidate image region (A) at the second position (x2, y2) in the second image (2) c ), to determine the candidate image region (A) c Does there exist an object in the image that is equivalent to the first object (9)? The second position (x2, y2) corresponds to the first position (x1, y1) according to the mapping between the coordinates in the thermal image (1) and the coordinates in the second image (2); In response to determining the candidate image region (A) c Since there is no equivalent object in (), the first object (9) is determined to be a reflection.

2. The method according to claim 1, wherein, Analyze the candidate image region (A) c This includes comparing the appearance of the first object (9) with the candidate image region (A). c The appearance of the candidate image region (A) is compared (S6) to determine the appearance of the candidate image region (A). c Does there exist an object in the first object (9) that is equivalent to the first object (9)? The method further includes: In response to (S7) the appearance of the first object (9) and the candidate image region (A) c If the difference in appearance exceeds a first threshold amount (δ1), it is determined (S8) that the first object (9) is a reflection.

3. The method according to claim 2, wherein, Appearance is derived from at least one of the group consisting of shape, texture, pattern, and contrast.

4. The method according to claim 2, wherein, Analyze the candidate image region (A) c This includes analyzing whether an object exists in the candidate image region to determine the candidate image region (A). c Does there exist an object in the first object (9) that is equivalent to the first object (9)? The method further includes: In response to (S7) determining the candidate image region (A) c If no object exists in (S8), determine that the first object (9) is a reflection.

5. The method of claim 1, further comprising: Detect another object at a third location (x3, y3) in the thermal image, and Among them, the analysis of the candidate image region (A) c )include: Detect the corresponding object in the fourth position (x4, y4) in the second image (2), the fourth position (x4, y4) and the third position (x3, y3) correspond to each other according to the mapping between the coordinates in the thermal image (1) and the coordinates in the second image (2); The appearance of the corresponding object is compared with the candidate image region (A). c The contrast value of the corresponding object is compared (S6) to determine whether the corresponding object is an equivalent object to the first object (9). The method further includes: In response to (S7) the appearance of the corresponding object and the candidate image region (A) c If the difference in appearance between the two objects exceeds a second threshold (δ2), it is determined (S8) that either the first object (9) or the other object is a reflection. Wherein, the appearance is a contrast value, and where, In response to (S7) the contrast value of the corresponding object exceeding the candidate image region (A) c If the contrast value of the object (9) exceeds the second threshold amount (δ2), it is determined (S8) that the first object (9) is reflective, and In response to the candidate image region (A) c If the contrast value of the other object exceeds the second threshold amount (δ2) of the contrast value of the corresponding object, it is determined that the other object is a reflection.

6. The method according to any one of the preceding claims, further comprising: In response to determining that the first object is a reflection, the reflection in the thermal image is compensated.

7. The method according to claim 6 when referring to a dependent claim 5, wherein, Compensating for the reflection in the thermal image (1) includes: It is determined that the other object is the source object, and the first object is a reflection of the source object; Determine the first pair of reference points (P) of the first object a1 P b1 The reflection intensity gradient (G) between ) r ); Determine the corresponding second pair of reference points (P) of the source object. a2 P b2 The source intensity gradient (G) between ) s ); Calculate the gradient ratio (R) between the reflection intensity gradient and the source intensity gradient. G ); For each pixel within the first object (9), the capture intensity value of the pixel is calculated by comparing it with the capture intensity value of the corresponding pixel in the source object and the gradient ratio (R). G The difference between the products of ) (Δ G To calculate the compensation strength value (I) comp ).

8. The method according to claim 6, wherein, Compensating for the reflection in the thermal image (1) includes: Interpolation is performed between the intensity values ​​of the pixels (10) surrounding the first object (9).

9. The method according to claim 6, wherein, Compensating for the reflection in the thermal image (1) includes: Store the background intensity value captured when there is no reflection at the location of the first object (12); For each pixel within the first object, the captured intensity value of the pixel is replaced with the pixel's stored background intensity value (12).

10. The method according to claim 6, wherein, Compensating for the reflection in the thermal image (1) includes: For each pixel within the first object (9), a representative pixel (13) outside the said position of the first object is selected, and the captured intensity value of the pixel is replaced with the captured intensity value of the representative pixel (13).

11. A system for detecting reflections in a first thermal image (1) captured by a thermal image sensor (31), the system (20) comprising: An image receiver (21) is configured to receive the thermal image (1) and the second image (2), wherein the second image (2) is captured by a visible light image sensor, a near-infrared sensor or a short-wave infrared sensor, and the thermal image (1) and the second image (2) have overlapping fields of view; The mapping module (22) is configured to determine a mapping from coordinates in the thermal image to coordinates in the second image; An object detector (23) is configured to detect a first object at a first location (x1, y1) in the thermal image; Image region analyzer (24) is configured to analyze candidate image regions (A) at a second location (x2, y2) in the second image. c ), to determine the candidate image region (A) c Does there exist an object in the image that is equivalent to the first object (9)? The second position (x2, y2) corresponds to the first position (x1, y1) according to the mapping between the coordinates in the thermal image (1) and the coordinates in the second image (2); The reflection determination module (26) is configured to respond to determining the reflection in the candidate image region (A) c Since there is no equivalent object in (), the first object (9) is determined to be a reflection.

12. The system according to claim 11 further includes a thermal camera (7) and a visible light camera (8), a near-infrared camera, or a short-wave infrared camera.

13. A thermal camera, comprising a thermal image sensor (31) and a system (20) according to claim 11.

14. A non-transitory computer-readable storage medium storing instructions that, when executed on a processing device, perform the method according to claim 1.

Images