Methods for generating thermal images

Abstract

A method for providing thermal image data using a device is provided, the device comprising: an IR sensor arranged to receive IR radiation from the surrounding environment in a field of view; and a position determination device capable of determining the position and orientation of the IR sensor, wherein the thermal image data represents a plurality of pixels in a dot matrix or bitmap, the method comprising the following steps: a) the IR sensor determines an IR sensor value corresponding to a temperature at at least a predetermined frequency, b) scanning the field of view over an area of ​​interest for which thermal image data is to be generated, c) the position determination device determines the respective current relative position and orientation of the device for each determination in step a) during the process of scanning the field of view, d) determining a thermal image of the area using the determined IR sensor value and the detected position and orientation to which the determined IR sensor value corresponds, wherein the temperature of a pixel in the image is determined by determining the temperature of two partially overlapping fields of view, such that the overlap between the two fields of view is associated with a non-common field (non-common field) of the two overlapping fields of view, and wherein the temperature value of the non-common field is determined by using a temperature difference between the two partially overlapping fields of view and a ratio of the area of ​​the non-common field relative to the area of ​​the fields of view, and wherein the temperature of a pixel in the thermal image data is determined by using the temperature value so determined.

Description

Technical Field

[0001] The present invention relates to systems and methods for collecting and processing information from IR sensors to produce thermal images, and a method of producing thermal image data. Background Art

[0002] Thermal imaging cameras are well known. Most thermal imaging cameras use lenses to focus IR radiation onto a chip with multiple sensors. Such sensors are expensive and bulky. A simple and low-cost thermal imaging camera is needed. Summary of the invention

[0003] In a first aspect of the invention, there is provided a method of providing thermal image data using a device, the device comprising: an IR sensor arranged to receive IR radiation from the surrounding environment in a field of view; and a position determination device capable of determining the position and orientation of the IR sensor, the method comprising the following steps:

[0004] a) the IR sensor determines an IR sensor value corresponding to the temperature at least at a predetermined frequency,

[0005] b) scanning the field of view over the region of interest for which thermal image data is to be generated,

[0006] c) the position determination means determines, during scanning of the field of view, for each determination in step a), the respective current relative position and orientation of the device,

[0007] d) determining thermal image data of the region of interest using the determined IR sensor values ​​and the detected position and orientation to which the determined IR sensor values ​​correspond,

[0008] Wherein the temperature of an area of ​​an image is determined by determining the temperature of two partially overlapping fields of view, such that the overlap between the two fields of view is associated with a non-shared field (non-shared field) of the two overlapping fields of view, and wherein the temperature value of the non-shared field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-shared field to the area of ​​the field of view.

[0009] In particular, a method for providing thermal image data using a device comprising: an IR sensor arranged to receive IR radiation from a surrounding environment in a field of view; and a position determination device capable of determining a position and orientation of the IR sensor, wherein the thermal image data represents a plurality of pixels in a dot matrix or bitmap, the method comprising the following steps:

[0010] a) the IR sensor determines an IR sensor value corresponding to the temperature at least at a predetermined frequency,

[0011] b) scanning the field of view over the region of interest for which thermal image data is to be generated,

[0012] c) the positioning determination means determines the respective current relative position and orientation of the device for each determination in step a) during the process of scanning the field of view,

[0013] d) determining a thermal image of the area using the determined IR sensor values ​​and the detected positions and orientations to which the determined IR sensor values ​​correspond,

[0014] wherein the temperature of a pixel in an image is determined by determining the temperature of two partially overlapping fields of view, such that the overlap between the two fields of view is associated with a non-shared field (non-shared field) of the two overlapping fields of view, and wherein the temperature value of the non-shared field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-shared field to the area of ​​the fields of view, and wherein the temperature of a pixel in thermal image data is determined by using the temperature value so determined.

[0015] The design with one sensor provides a cost-effective way to provide thermal imaging in low-cost devices with a single IR sensor, such as mobile phones.

[0016] In one embodiment, it is assumed that the entire temperature difference between two partially overlapping fields is caused by one or both of the non-common fields. In one embodiment, it is assumed that the entire temperature difference between two partially overlapping fields is caused by one of the non-common fields.

[0017] The field of view can be scanned in at least two different scanning directions, thereby generating a first temperature value of the non-shared field from a first scanning direction and a second temperature value of the non-shared field from a second scanning direction, and the first temperature value and the second temperature value are used to determine the temperature of a region in the image or a pixel representing a region shared by two non-shared fields. Using multiple scanning directions increases the resolution of the image. Adding more scanning directions can increase the resolution.

[0018] Data from different scan directions may be used in any suitable manner to determine the temperature of the shared area.In one embodiment, the temperature of a region or pixel in the image is determined by averaging the temperature of the shared area.

[0019] The IR sensor may be provided in a handheld device, such as a mobile device such as a mobile phone. Instructions may be provided to the user to scan the device in at least two different directions that are not detection directions of the field of view of the IR sensor. The instructions to the user may include instructions to move the handheld device in a predefined direction, and wherein the specific direction is updated based on data from the position and orientation determination device. When the device has a display, the display may be used to display instructions prompting the user to scan, and also to display a thermal image created using thermal image data. The display may be used to display the image in real time as the image is being formed. This has the advantage of providing the user with feedback on how to scan in real time to improve the image.

[0020] In a second aspect of the present invention, a system is provided, the system comprising: an IR sensor, the IR sensor being arranged to receive IR radiation from the surrounding environment in a field of view; and a positioning determination device, the positioning determination device being capable of determining the position and orientation of the IR sensor, the system being arranged to provide thermal image data, wherein the system is configured to cause the IR sensor to determine an IR sensor value corresponding to a temperature at at least a predetermined frequency, and is further configured to determine a current relative position and orientation of the IR sensor for a point in time at which the IR sensor value is determined, and is further configured to determine data of the thermal image using the determined IR sensor value and the detected position and orientation corresponding to the determined IR sensor value, wherein the temperature of an area in the image is determined by determining the temperature of two partially overlapping fields of view, so that the overlap between the two fields of view is associated with two fields (non-shared fields) that are not shared by the two overlapping fields of view, and wherein the temperature value of the non-shared field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-shared field to the area of ​​the field of view.

[0021] In particular, a system is provided, the system comprising: an IR sensor, the IR sensor being arranged to receive IR radiation from the surrounding environment in a field of view; and a position determination device, the position determination device being capable of determining the position and orientation of the IR sensor, the system being arranged to provide thermal image data representing a plurality of pixels in a dot matrix or bitmap, wherein the system is configured to cause the IR sensor to determine an IR sensor value corresponding to a temperature at at least a predetermined frequency, and further configured to determine a corresponding current relative position and orientation of the IR sensor for a point in time at which the IR sensor value is determined, and further configured to determine data of the thermal image using the determined IR sensor value and the detected position and orientation corresponding to the determined IR sensor value, wherein the temperature of a pixel in the image is determined by determining the temperature of two partially overlapping fields of view, such that the overlap between the two fields of view is associated with two fields not shared by the two overlapping fields of view (non-shared fields), and wherein the temperature value of the non-shared field is determined by using a temperature difference between the two partially overlapping fields of view and a ratio of an area of ​​the non-shared field relative to an area of ​​the field of view, and wherein the temperature of a point in the image is determined by using the temperature value so determined for the non-shared field to which the point in the image belongs.

[0022] In a third aspect of the present invention, there is provided a software arranged to repeatedly obtain IR sensor values ​​from an IR sensor, and also arranged to repeatedly obtain the position and orientation of the IR sensor from a positioning determination device, the software also being configured to determine a thermal image of the area using the IR sensor values ​​and the detected position and orientation corresponding to the IR sensor values, wherein the temperature of the area in the image is determined by determining the temperature of two partially overlapping fields of view, so that the overlap between the two fields of view is associated with a field that is not common to the two overlapping fields of view (non-common field), and wherein the temperature value of the non-common field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-common field to the area of ​​the field of view, and wherein the temperature of the area in the thermal image data is determined by using the temperature value so determined.

[0023] In particular, a software is provided, which is arranged to repeatedly obtain IR sensor values ​​from an IR sensor, and is also arranged to repeatedly obtain the position and orientation of the IR sensor from a positioning determination device, and the software is also configured to use the IR sensor values ​​and the detected position and orientation corresponding to the IR sensor values ​​to determine a thermal image of an area, wherein the thermal image data represents a plurality of pixels in a dot matrix or a bitmap, wherein the temperature of a pixel in the image is determined by determining the temperature of two partially overlapping fields of view, so that the overlap between the two fields of view is associated with a field that is not common to the two overlapping fields of view (non-common field), and wherein the temperature value of the non-common field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-common field to the area of ​​the fields of view, and wherein the temperature of the pixel in the thermal image data is determined by using the temperature value so determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of the system.

[0025] Figure 2 is a schematic diagram of the device.

[0026] Figure 3 is a schematic diagram of a thermal image.

[0027] Figures 4 to 7 is a schematic diagram of the device and field of view.

[0028] Figure 8 is a flow chart of the method.

[0029] Figures 9 to 11 is a schematic image of the field of view.

[0030] Fig.12 is a flow chart of the method.

[0031] Figure 13 to Figure 14 is a schematic diagram of the field of view and the region of interest.

[0032] Figure 15 to Figure 16 is a schematic diagram of a device with a display.

[0033] Figures 17 to 24 Experimental data are shown. DETAILED DESCRIPTION

[0034] refer to Figures 1 to 6, the system 1 comprises an IR sensor 2 arranged to receive IR radiation in a field of view 4. The field of view 4 has a detection direction 5. The IR sensor 2 may be any useful type of IR sensor, such as a thermopile or a bolometer, wherein thermopiles are preferred due to their smaller response time. Preferably, the IR sensor is a type of sensor providing absolute temperature values, such as a thermopile. The angle α of the field of view 4 may be from 5° to 120°, wherein preferably it is from 6° to 45°, more preferably from 7° to 15°. The field of view 4 may have any suitable shape, preferably circular, such as Figure 3 , Figures 5 to 7 , Figures 9 to 11 shown.

[0035] The IR sensor 2 preferably senses one value and only one value at a given moment. Thermal image data is generated by scanning the field of view 4 of the IR sensor 2 over the area of ​​interest 3 and repeatedly collecting IR information during the scan, and then compiling and processing the collected information using the position and / or orientation of the IR sensor 2 to produce image data.

[0036] The system 1 also includes a memory 8, a processor 9 and a communication bus 10. In a preferred embodiment, the system 1 includes a digital display 11 for displaying digital images. The digital display 11 can be, for example, an LCD display. The system 1 can include any suitable combination of software and hardware. The system 1 can, for example, include an operating system and a device for input, such as a touch display.

[0037] The memory 8 is used to store data, for example software, for example implementing the method, and also necessary data, such as IR sensor values, positions, orientations, time points, time stamps, overlapping fields and their positions, and image data.

[0038] The system 1 further comprises a positioning and / or orientation determining device 6, herein referred to as "position determining device 6", capable of determining the position and / or orientation of the IR sensor 2. The orientation of the IR sensor 2 means the orientation of the detection direction 5. Preferably, both the position and the orientation are determined by the position determining device 6, but in some embodiments only the position or the orientation is determined. The position and the orientation are determined with a fine resolution, preferably such that movements of a few millimeters (for translation) or a few degrees (for orientation) can be detected.

[0039] The position and orientation are determined using methods known in the art. Thus, the position determining means 6 may include any suitable technology, including tracking cameras, marker-based or markerless tracking, inertial tracking, digital model building, gyroscopes and accelerometers, which may be combined in any suitable manner. Current models of mobile phones (such as the iPhone X and Android phones) are capable of determining position and orientation with high resolution. Apple iOS provides an API called Core Motion for providing position and orientation data, and Android also provides a similar API.

[0040] Preferably, the IR sensor 2 and the position determination means 6 are included in a device 7, which is preferably a handheld device. The device 7 may be a mobile phone, such as an iPhone or an Android phone. However, parts of the system 1 may be located outside the device 7, such as on a server. In particular, the memory 8 and the processor 9 or parts of the memory 8 and the processor 9 may be located on a server that is in digital communication (e.g. wireless communication) with the device 7.

[0041] The detection direction 5 of the field of view 4 is preferably fixed relative to the device 7. The position determination means 6 is capable of determining the detection direction 5 of the field of view 4 and the position of the IR sensor 2. The position and direction 5 are preferably determined relative to objects in the surroundings, such as objects in the region of interest 3. The position and direction may preferably be determined relative to stationary objects. The detection direction 5 may be identical to a viewing line which is perpendicular and points to the surface of the display 11 on the device 7 (see Figure 5 and Figure 6 ). A user holding the device 7 with the display 11 in his hand will observe the display 11 in the same direction as the detection direction 5 of the IR sensor 2, which is advantageous during real-time display of thermal images (see below).

[0042] The system 1 may also include suitable components known in the art for processing the signal from the IR sensor 2, such as amplifiers, filters, A / D converters, etc. The system 1 is powered by a suitable power source, such as a battery. The IR signal is provided from the IR sensor 2 to the rest of the system 1, which is capable of processing the signal, storing data representing the signal, and using this data to perform calculations using any suitable combination of software and hardware.

[0043] The system 1 is used to generate thermal image data in the form of a matrix of dots 12 representing a thermal image, an example of which is shown in FIG. Figure 3. The dot matrix 12 includes pixels 13 constituting an image. The dot matrix image may be a bitmap image. The image data is suitable for presenting the thermal image 12 on a display (e.g., the display 11 of the system 1). The image data may be arranged as a dot matrix 12, wherein a temperature value is stored for each pixel 13. The number of pixels in the thermal image 12 may range from, for example, 10,000 to several million.

[0044] Each pixel 13 in the dot matrix 12 can adopt at least two different colors (for example black and white), preferably more colors. Each color represents a temperature value or a range of temperature values. The color scheme is selected according to the application. Typically, thermal images use a color scale: black-blue-green-yellow-red, where black indicates the lowest temperature and red indicates the highest temperature. Each color indicates a specific temperature range. These colors are pseudo-colors, and any useful color scheme can be used. The temperature range of each color is selected according to the intended use. The display 11 of the device 7 can be used to present a thermal image based on the dot matrix 12. Therefore, the pixel 13 can display a color representing a temperature value.

[0045] To generate image data, the field of view 4 is scanned over the region of interest 3 (i.e., the region where the thermal image 12 is to be generated). The scanning direction is indicated in the figure by arrow 15. When the device 7 is held in the hand, the scanning is performed manually by moving the hand. However, in some embodiments, powered and / or automatically controlled scanning can be used, whereby the field of view 4 is moved by a motor controlled by a computer, for example. Therefore, the system 1 and method described herein can be used as an alternative to a multi-pixel thermal imaging chipset.

[0046] The IR sensor 2 determines at least a predetermined frequency the IR value in the field of view 4. For a handheld device, a suitable frequency may be from 1 Hz to 100 Hz, preferably 2 Hz to 20 Hz.

[0047] The position determination device 6 repeatedly determines the position and / or orientation of the IR sensor 2 at appropriate time intervals, which can be performed simultaneously with the capture of the IR data or with the same frequency. However, different sampling frequencies can be used. The sampling frequency of the position and / or orientation can be at least the frequency at which the IR data is sampled. A time matching algorithm can be used to match the correct IR data with the correct data of the position and / or orientation. For example, a timestamp can be used. Therefore, during the process of scanning the field of view 4, the position determination device 6 detects the corresponding current relative position and / or orientation of the IR sensor 2 for each IR value.

[0048] The direction of scanning the field of view 4 can be accomplished by translating the field of view 4, for example by translating the device 7 so that the field of view 4 moves across the region of interest 3 ( Figure 5 , where the translation direction is indicated by arrow T), or by changing the detection direction 5, for example Figure 6 This is shown by rotating the device 7 (ie turning the device 7 about an axis not corresponding to the detection direction 5, the direction of rotation being indicated with arrow R), or a combination of these. In a preferred embodiment, both translation and direction change are used.

[0049] Thus, each IR value detected during scanning is associated with data representing a position and / or orientation. IR image data can be compiled by using the temperature values ​​and positioning them in the dot matrix 12 using the position and / or orientation data. The position and / or orientation data obtained by the position determination device 6 can be used to position the various captured fields of view in three-dimensional space. Figure 3 As shown, the field of view 4 (or part of the field of view, see below) can assign temperature values ​​to multiple pixels 13. Therefore, a pixel 13 in the image 12 generally corresponds to a smaller area than the field of view 4. More than one pixel 13 can be used to represent an area for which a temperature value has been determined, and one pixel can be used to represent more than one area for which a temperature has been determined.

[0050] Sampling can be used to map various areas where the temperature has been determined to pixels 13 of the image 12. For example, one or more pixels 13 in the image 12 can be used to depict the shared area 21 (see below). Any suitable bitmap digital image file format can be used, including but not limited to JPEG, TIFF or GIF. Vector graphics can also be used, although it seems simpler to use a bitmap / dot matrix file format.

[0051] Figures 7 to 9 A preferred embodiment is shown. Here, the sampling speed and the scanning speed are such that there is a partial overlap between the fields of view 4a and 4b. The two fields of view 4a, 4b are associated with two fields 102 and 103 which are not common to the two overlapping fields of view 4a, 4b.

[0052] exist Figure 7 , the scanning direction is again indicated by arrow 15. The "old" non-shared field 102 is a portion of the field of view 4a captured before the field of view 4b. The "new" non-shared field 103 is a portion of the field of view 4b captured after the field of view 4a. In various embodiments, different assumptions are made as to what affects any detected changes in temperature between the area 4a and the area 4b. The difference in temperature between the old field of view 4a and the new field of view 4b can be attributed to the old non-shared field 102 or the new non-shared field 103, or both. The temperature value of at least one non-shared field 102 and 103 is determined by using the temperature difference between the two partially overlapping fields of view 4a and 4b and the ratio of the area of ​​the non-shared fields 102 and 103 relative to the area of ​​the fields of view 4a and 4b. The temperature of a pixel 13 in the image 12 is determined by using the temperature value so determined for at least one non-shared field 102, 103 to which the pixel 13 belongs.

[0053] The ratio of the area of ​​the non-common parts 102, 103 relative to the field of view 4a, 4b depends on the sampling frequency and the speed at which the scanning is carried out. The scanning speed can be varied, in particular when using a handheld device 7. The ratio of the area of ​​the non-common fields 102, 103 relative to the area of ​​the field of view 4 can be determined by an area ratio determination device. The area ratio determination device can use the current scanning speed determined by the position determination device 6. The area ratio determination device is part of the system 1 and is preferably implemented as software that uses information from the position determination device 6 and the sampling time points. Therefore, the area ratio determination device uses the information from the position determination device 6 to determine the speed at which the field of view 4 moves over the region of interest 3, and can use this information to determine the ratio between the area of ​​the non-common fields 102, 103 relative to the fields of view 4a, 4b. Higher scanning speeds result in larger non-common fields 102 and 103, and slower scanning speeds result in smaller non-common fields. Low scanning speeds result in improved resolution of thermal information. A suitable ratio of the non-common field relative to the total field of view 4 can be 1% to 20%.

[0054] In one embodiment, when the scanning speed is high, the sampling frequency of the IR sensor and the position determination device 6 is automatically increased.

[0055] The change in temperature between the fields of view 4a and 4b is divided by the area ratio, and the resulting ratio is used to calculate the temperature of the old and / or new non-shared fields 102 and 103. In one embodiment, it is assumed that the entire temperature difference between the overlapping fields 4a, 4b is due to the new non-shared field 103. In one embodiment, it is assumed that the entire temperature difference is due to the old non-shared field 102. In other embodiments, it is assumed that the temperature difference is due to both the new non-shared field 103 and the old non-shared field 102, for example in a fixed ratio, such as 50% due to the new non-shared field 102 and 50% due to the old non-shared field 103.

[0056] This is the case in the following, where it is assumed that the entire change in temperature from field 4a to field 4b is due to the new non-shared field 103. For example, assume that the temperature of field 4a is 23° C., and the temperature of area 4b is 21° C. The difference in temperature (-2° C.) will be due to the non-shared fields 102, 103.

[0057] If we assume that the unshared field 103 has 15% of the size of the field of view 4b, there will be a change of -2°C / 0.15 = -13.3°C, ie the temperature value of the new unshared field 103 will be 23-13.3 = 9.7°C. Using data from more scan directions can improve accuracy as described below.

[0058] Figure 8The method is shown in FIG. 2. In step 200, IR sensor data from two overlapping fields of view 4a and 4b from the same scan direction are collected. In step 201, the ratio of the area of ​​the non-shared fields 102 and 103 to the fields of view 4a and 4b is determined by an area ratio determination device. Note that the ratio between the non-shared field 102 and the field of view 4a will be the same as the ratio of the non-shared field 103 to the field of view 4b. Steps 200 and 201 can be performed in any order. In step 202, the temperature of the non-shared field 102 or 103 or both is determined.

[0059] The method is preferably applied to a series of overlapping fields of view 4a, 4b, 4c, 4d, such as Fig. 9 As shown, the overlap between 4a and 4b is used, the overlap between 4b and 4c is used, the overlap between 4c and 4d is used, etc. In this way, most of the area of ​​interest 3 can be covered by non-shared fields. The starting point 14 can be used to relate the position and orientation of the different fields of view 4a, 4b and their non-shared fields 102, 103. The starting point 14 is preferably assigned to the first sampled field of view 4a. The position determination device 6 is capable of building a model of how the field of view 4 moves and mapping the IR sensor values ​​relative to the starting point 14 to create the thermal image 12. The starting point 14 is preferably a point in space defined relative to the surrounding environment or the area of ​​interest 3. The starting point 14 is preferably associated with the initial viewing direction 5.

[0060] A larger region of interest 3 can be covered by scanning the region of interest 3 multiple times, each time with a suitable offset ( Fig.10 ). The offset should preferably be at most the diameter of the field of view 4.

[0061] refer to Figure 11 to Figure 12 , preferably the field of view 4 is scanned over the region of interest 3 in at least two different directions 15a, b. For example, a point 20 is covered by the new non-shared field 103a from a first direction 15a and by the new non-shared field 103b from a second direction 15b. The temperature of the point 20 or region 21 is determined as described above by using the temperature values ​​of the first and new second non-shared fields 103a, b, and then the temperature values ​​of the point 20 or region 21 from the at least two scanning directions 15a, 15b are used to determine the temperature value of a region in the thermographic image 12 (such as a pixel 13 or a plurality of pixels in the thermographic image 12).

[0062] In step 300, the field of view is scanned over the region of interest 3 in a first direction 15a, thereby generating a first temperature value of a first new non-common field 103a. This is done using the method described above with reference to Figures 7 to 10The method is completed. In step 301, the field of view 4 is scanned over the region of interest 3 in the second direction 15b, thereby generating a second temperature of the second old non-shared field 103b in the same way. In step 303, it is determined that the first and second old non-overlapping fields 103a, b cover the same area (shown as shared area 21) to a certain extent. The first and second temperature values ​​are used to determine the temperature of the shared area 21 and can be used to determine the temperature value of the pixel 13 in the shared area 21 of the image 12. The shared area 21 (dashed line) will have the same temperature value.

[0063] The shared area 21 is determined by using the position information stored for each non-shared area 103a, 103b. For example, a starting position 14 may be used and all temperature values ​​may be spatially associated with the starting position 14.

[0064] It is described above how to use new non-shared regions 103a,b, but old non-shared regions 102a,b may be used instead. As described above, both new and old non-shared fields may also be used, where for example any temperature variation is split between new and old regions (e.g. in a predetermined manner), e.g. 50 / 50.

[0065] In a preferred embodiment, values ​​from three, four, five, six, seven, eight or more scanning directions 15 are used. Therefore, temperature values ​​of three, four, five, six, seven, eight or more non-overlapping areas 102, 103 can be used. Using temperature data from multiple (such as two or more) scanning directions increases the resolution of the thermal image 12. The temperature data from multiple scanning directions can be used in any suitable manner. In one embodiment, the average value of temperature values ​​from different directions 15ab is used. In one embodiment, when there are more than two scanning directions, the value farthest from the average value is used. In one embodiment, the extreme value closest to the average value is used (if the extreme values ​​are 10 and 2 and the average value is 5, then 2 is selected). In one embodiment, the median value is used.

[0066] Thus, for each pixel 13 in the image 12, the temperature value may be a result of using temperature data of a plurality of temperature values, such as an average of the plurality of temperature values.

[0067] The scanning directions 15a and 15b of the two data sets are preferably separated by at least 45°. Preferably, the two scanning directions are separated by 90°. In a preferred embodiment, the field of view 4 is scanned in at least four mutually different directions 15, thereby generating four different sets of determined IR sensor values. The mutually different directions 15 are preferably separated by 90°. In a more preferred embodiment, eight different directions 15 separated by 45° are used. In one embodiment, three different scanning directions 15 separated by 120° are used.

[0068] The scanning is preferably carried out in the same plane or in parallel planes. These planes are preferably perpendicular to the viewing direction 5 of the field of view 4 at the starting point 14 .

[0069] exist Fig.13 In one embodiment shown in FIG. 1 , the field of view 4 is scanned several times in a first direction 15a with a vertical offset, so that each scan at least partially covers a different area, and wherein the field of view 4 is additionally scanned several times in a second direction 15b with a different vertical offset, so that each scan 16 covers at least partially a different area of ​​the region of interest 3. Each arrow 16 indicates the path of the centroid of the projection of the field of view 4 when it covers the region of interest 3.

[0070] However, random scans can be used, such as Fig.14 As shown. For example, a user holding the device 7 in his hand can scan the field of view 4 in a cross-pattern over the area of ​​interest 3 to collect information from several directions. In one embodiment, the image is displayed in real time on the display 11, providing feedback to the user as the image improves during the scan. Thus, the user is able to create a thermal image using a "painting" type of motion. This is particularly useful when the image 12 is shown in real time on the display 11, see below. The user can then improve the image 12 where needed by scanning over the area of ​​interest 3 that needs improvement, and see the results in real time.

[0071] As described above, the user may be provided with instructions to scan in different predefined directions (e.g., two directions separated by 90°). The instructions may be provided in any manner, such as text, symbols, or sounds (such as synthesized speech). In a preferred embodiment, the instructions are provided on the display 11. Fig.15 An example is shown in FIG. For example, a handheld device 7 is used by a user, and the user is prompted to move the field of view 4, for example by moving the device 7 in different directions 15a, 15b. In a preferred embodiment, the device 7 has a display 11, and the display 11 is used to prompt the user. The display 11 can be used to indicate to the user the direction 15 in which the user should scan the device 7. The display 11 can also provide guidance for orienting the device 7 and therefore the viewing direction 5 in the correct direction. As described above, the prompts can be provided simultaneously when the real-time image is displayed.

[0072] exist Fig.16 In one embodiment shown in FIG, information from the position determination device 6 is used to provide instructions to the user in real time. In particular, the user may be prompted to translate or rotate the device 7 by translating or rotating the field of view 4 in order to maintain a predefined scanning direction and orientation. Fig.14In the embodiment, the display 11 shows an arrow 17 and a level, which includes a fixed cross 18 and an orientation-related cross 19, which moves on the display 11 as the orientation of the device 7 is determined by the orientation determination device 6. The user is prompted to keep the orientation-related cross 19 aligned with the fixed cross 18. The arrow 17 moves to instruct the user to scan in different directions. Therefore, the arrow 17 can be controlled by the position determination device 6 and a predetermined movement pattern, for example Fig.13 For example, the user may be prompted to first scan in a first direction 15a with an offset, and then scan in a second direction 15b, where the directions are separated by an angle as described above. The position and orientation at the starting point 14 may be used to provide guidance to the user, where the user is prompted to hold the device 7 in the same orientation or position as at the starting point 14.

[0073] The user may also be prompted to scan in a certain plane, which may be a plane perpendicular to the viewing direction 5 at the starting point 14, or the plane of the display 11 of the device 7 at the starting point 14. Commands such as "closer" and "further" may be used on the display 11 to keep the device 7 in the plane while scanning.

[0074] The user can also be prompted to scan at a certain speed. As mentioned above, if the scanning speed is low, the resolution will be better. Therefore, the user can be prompted to increase or decrease the scanning speed according to the situation. Feedback can be provided in real time.

[0075] In one embodiment, the display 11 of the device 7 is used to display a thermal image 12 using the data of the thermal image processed as described above. Thus, the method may include the step of displaying the thermal image 12 of the display 11 of the device 7 including the IR sensor 2. The image 12 may be reproduced in real time on the display 11 while the user scans over the area of ​​interest 3. Thus, the image data may be provided to the display 11 for real time reproduction. This provides feedback to the user.

[0076] Imaging software for implementing the methods described herein is also provided. The imaging software can be provided as an application to be downloaded to a portable device 7 having an IR sensor 2 and a position determination device 6. The imaging software is preferably stored on a memory 8. The imaging software can be configured to obtain IR sensor values ​​and position and / or orientation data from the position determination device 6. For example, the imaging software can repeatedly query one or more sensors, or turn on one or more sensors, and provide a predetermined configuration to one or more sensors. Such a configuration can be, for example, an instruction to the IR sensor 2 or the position determination device 6 regarding the sampling frequency. The imaging software can be configured to pair the IR sensor data with the position and / or orientation data. The imaging software includes the above-mentioned ratio determination device.

[0077] The imaging software is configured to compile the data to produce thermal imaging data, thereby producing the thermal image 12. Therefore, the imaging software is configured to determine non-overlapping fields and the positions and orientations of these fields, and compile temperature data from different directions as described herein.

[0078] Example 1

[0079] The method of the present invention was simulated using digital thermal data of a region of interest showing the Figures 17 to 24 The person and dog are shown in the upper left inset of . The data was scanned in eight different directions (indicated by arrows) using a virtual field of view that scans the digital image data. The position of the virtual field of view is controlled by the user using a computer mouse. Overlapping fields are determined and any difference in temperature between two overlapping fields of view is 100% attributed to the new non-shared field. For each pixel in the image, the temperatures of up to eight non-overlapping fields are averaged. The upper right figure shows the result of averaging the data set. It can be seen how the image improves as more and more scan directions are added. At the bottom right is an image showing only the average temperature in the field of view (without using overlap and different directions), which clearly provides poorer resolution.

Claims

1. A method for providing thermal image data using a device, the device comprising: an IR sensor arranged to receive IR radiation from the surrounding environment in the field of view; and a position determination device capable of determining the position and orientation of the IR sensor, the method comprising the steps of: a) the IR sensor determines an IR sensor value corresponding to the temperature of the field of view at least at a predetermined frequency, b) scanning the field of view over a region of interest for which thermal image data is to be generated, wherein the scanning is performed such that two consecutive IR sensor values ​​determine corresponding fields of view that have overlap, c) the position determination means determines, during scanning of the field of view, for each determination in step a) a respective current relative position and orientation of the device, d) determining data of a thermal image of the region of interest using the determined IR sensor values ​​and the corresponding detected positions and orientations of the determined IR sensor values, wherein the temperature of the area in the image is determined by determining the temperature of two partially overlapping fields of view, such that the overlap between the two fields of view is associated with two non-shared fields, which are non-shared fields of the two overlapping fields of view, and wherein the temperature value of the non-shared field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-shared field to the area of ​​the field of view.

2. The method according to claim 1, wherein: It is assumed that the entire temperature difference between the two partially overlapping fields is caused by one or both of the non-common fields.

3. The method according to claim 1 or 2, wherein: It is assumed that the entire temperature difference between the two partially overlapping fields is caused by one of the non-shared fields.

4. The method according to claim 1 or 2, wherein: The field of view is scanned over the region of interest in at least two different scanning directions, thereby generating a first temperature value of the non-common field from a first scanning direction and a second temperature value of the non-common field from a second scanning direction, and the first temperature value and the second temperature value are used to determine the temperature of an area in the image, the area representing an area shared by two non-common fields.

5. The method according to claim 4, wherein: The temperature of a region in the image is determined by averaging the temperatures of the shared regions.

6. The method according to claim 1, wherein: The IR sensor is included in a handheld device.

7. The method according to claim 6, wherein: Instructions are provided to the user to scan the device in at least two different directions that are not detection directions of the field of view of the IR sensor.

8. The method according to claim 7, wherein: The instructions to the user include instructions to move the device in a predefined direction, and wherein the predefined direction is updated based on data from the position determining means.

9. The method according to any one of claims 7 to 8, wherein: The device has a display, and wherein the display is used to display an instruction prompting the user to scan, and to display a thermal image created using the thermal image data.

10. The method according to claim 1 or 2, wherein: The device has a display, and wherein the display is used to display the image in real time as the image is formed.

11. A system for providing thermal image data, comprising: an IR sensor arranged to receive IR radiation from the surrounding environment in the field of view; and a position determination device capable of determining a position and an orientation of the IR sensor, the system being arranged to provide the thermal image data, wherein the system is configured to cause the IR sensor to determine an IR sensor value corresponding to the temperature of the field of view at least at a predetermined frequency, wherein the predetermined frequency is such that when the field of view is scanned over a region of interest, two consecutive IR sensor values ​​determine corresponding fields of view having overlap, and is further configured to determine a current relative position and orientation of the IR sensor for a point in time at which the IR sensor values ​​are determined, and is further configured to determine data of the thermal image using the determined IR sensor values ​​and the corresponding detected position and orientation of the determined IR sensor values, wherein the temperature of an area in the image is determined by determining the temperature of two partially overlapping fields of view, such that an overlap between the two fields of view is associated with two non-common fields, the two non-common fields being fields that are not common to the two overlapping fields of view, and wherein the temperature value of the non-common field is determined by using a temperature difference between the two partially overlapping fields of view and a ratio of an area of ​​the non-common field relative to an area of ​​the field of view.

12. A memory having software stored thereon, the software being for performing the method according to claim 1 when executed on a system according to claim 11, the software being arranged to repeatedly obtain IR sensor values ​​from an IR sensor and further arranged to repeatedly obtain the position and orientation of the IR sensor from a position determination device, the software being further configured to determine a thermal image of the area using the IR sensor values ​​and the corresponding detected position and orientation of the IR sensor values, wherein The temperature of the area in the image is determined by determining the temperature of two partially overlapping fields of view, so that the overlap between the two fields of view is associated with two non-common fields, which are fields that are not common to the two overlapping fields of view, and wherein the temperature value of the non-common field is determined by using the temperature difference between the two partially overlapping fields of view and the ratio of the area of ​​the non-common field to the area of ​​the field of view, and wherein the temperature of the area in the thermal image is determined by using the temperature value so determined.

Images