Method, program, apparatus, medium, and system for reducing noise in image data

Abstract

The invention relates to a method for reducing noise in image data, comprising the following steps: providing image data, the image data comprising at least one thermally imaged image, the image data being generated by detection by at least one sensor, a temperature difference exhibiting in the at least one thermal image is determined on the basis of an analysis of the temperature data of the at least one thermal image, and a number for averaging is determined on the basis of the determined temperature difference in order to evaluate the further image data, the number for averaging specifies how many thermal images are averaged in each case in order to evaluate further image data, which is generated by further detection by the at least one sensor, noise in the further image data being reduced by using the evaluation made by the determined number for averaging. The invention further relates to a computer program, a device, a storage medium and a camera system for this purpose.

Description

Technical Field

[0001] The invention relates to a method for reducing noise in image data. Furthermore, the invention also relates to a computer program, a device and a storage medium for this purpose. Background Art

[0002] In thermal images, increased noise occurs due to various factors, which are attributed, for example, to sensor properties. The detectors in thermal imaging cameras are highly sensitive and can detect even the smallest temperature differences themselves. However, this sensitivity also makes these detectors susceptible to electrical noise generated in the sensor itself. In addition, noise can also be a disturbing factor when reading the sensor. When digitizing analog signals, quantization errors may occur, which appear as noise. In particular, high-frequency and random noise can present challenges in the context of thermal images. In order to reduce noise in the image, it is known to average, for example, areas of image processing or motion recognition in the image. Summary of the invention

[0003] The subject matter of the invention is a method for reducing noise in image data, a computer program, a device for data processing, a computer-readable storage medium and a camera system. Further features and details of the invention are derived from the corresponding developments, the description and the drawings. Features and details described in conjunction with the method according to the invention are of course also applicable in conjunction with the computer program according to the invention, the device according to the invention, the computer-readable storage medium according to the invention and the camera system according to the invention, and vice versa, respectively, so that reference is always made to one another or can be made to one another in relation to the disclosure of the individual inventive aspects.

[0004] The subject matter of the invention is, in particular, a method for reducing noise in image data, which method comprises the following steps, wherein these steps can be performed repeatedly and / or successively.

[0005] In a first step, image data are preferably provided, wherein the image data comprises at least one thermal image. In particular, the image data is generated by detection by at least one sensor. Here, the at least one sensor is preferably a thermal image camera sensor, for example comprising an infrared detector array, so that the image data is in particular thermal image data. Thus, providing the image data may be providing thermal image data. The thermal image may be an infrared image.

[0006] In another step, the temperature difference displayed in the at least one thermographic image is preferably determined based on the analysis of the temperature data of the at least one thermographic image. Therefore, the temperature difference is particularly reflected in the range between the minimum temperature value and the maximum temperature value displayed in the thermographic image. The temperature difference is preferably determined for the corresponding single thermographic image. The temperature data is particularly the individual temperature values ​​represented by the corresponding colors in the thermographic image. In addition, the so-called span can also be determined based on the temperature difference in the image, which is specific to the representation of the temperature data in the form of color values ​​in the thermographic image. For example, a minimum span of 5°C can be defined, wherein the span is the difference between the minimum temperature and the maximum temperature displayed in the image. The span can correspond to the value of the temperature difference after exceeding the defined minimum span. In addition, it is also conceivable to evaluate the temperature distribution based on a histogram, wherein, for example, not all temperature values ​​are considered, but only 98% of the internal temperature values ​​are considered.

[0007] In another step, preferably based on the temperature difference obtained, a number for averaging is determined to evaluate other image data. It is also conceivable to determine the number for averaging based on the span obtained based on the temperature difference. The number for averaging is particularly given, how many thermographic images are averaged to evaluate other image data. The other image data is generated by another detection of at least one sensor, that is, in particular by another detection of the same at least one sensor that generates the image data. Therefore, the other image data is also preferably thermographic image data. In the case where the number of averages is 5, for example, 5 corresponding temperature values ​​from 5 thermographic images can be averaged for corresponding pixels or for corresponding pixel areas. In this case, it can be particularly provided that, compared with smaller temperature differences, a smaller number for averaging is provided in the case of larger temperature differences. The number for averaging can also correspond to a value of 1, wherein averaging is not provided, but rather provided for a single evaluation of the corresponding thermographic image.

[0008] Furthermore, it can be provided that the image update rate of the camera system according to the invention is influenced based on the number determined for averaging, or a change in the image update rate is initiated. For example, if the averaging is determined as a number of 2, the image update rate can be doubled.

[0009] In a further step, noise in the further image data is preferably reduced by evaluation using the determined number for averaging. The noise reduction is in particular a consequence of averaging, since by averaging, outliers in the individual thermographic images advantageously have a lower amplitude. The underlying principle of the noise reduction is in particular that a plurality of temporally sequential temperature data are detected for a corresponding pixel or pixel region in the thermographic image and the average of these temperature data is calculated. By averaging, occasional noise components that vary between the thermographic images can advantageously be reduced.

[0010] This can be expressed in mathematical terms as follows:

[0011]

[0012] Here, x i is the ith temperature data corresponding to pixel or pixel region x, and N is the number of thermal imaging images included in the average. The efficiency of noise suppression can be improved by increasing the value of N, naturally at the expense of the temporal resolution of the signal.

[0013] Advantageously, provision can be made in the invention that determining the quantity for averaging comprises the following steps:

[0014] - defining at least two regions for the temperature difference, wherein the defined quantity for averaging is assigned to the corresponding region,

[0015] - Based on a comparison of the ascertained temperature difference with at least two regions defined for the temperature difference, a quantity for averaging is determined.

[0016] For example, a region may be defined for a temperature difference of at least 10° C., to which the averaging of more than two thermographic images is assigned, and another region may be defined for a temperature difference of less than 10° C., to which the averaging of more than four thermographic images is assigned. Accordingly, for example, when a temperature difference of 12° C. is determined, the number of regions for averaging may be determined to be 2. Advantageously, the number of regions may vary according to the current application, for example, at least three regions may also be defined.

[0017] Optionally, it may be provided that the method further comprises the following steps:

[0018] The further image data are evaluated, wherein the corresponding generated thermographic images are ascertained based on the variables used for averaging.

[0019] The ascertained resulting thermographic image therefore corresponds in particular to the result of an averaging of a certain number of thermographic images.

[0020] According to another advantage, it can be provided that the method further comprises the following steps:

[0021] - Initiate output of the corresponding generated thermal imaging image.

[0022] The output can be carried out, for example, by means of an output unit which can be designed as a screen. The screen can be arranged in the camera system according to the invention, which can also have at least one sensor, ie preferably a thermal imaging camera sensor, which comprises, for example, an infrared detector array.

[0023] Optionally, it is also conceivable that the evaluation of the further image data comprises the following steps:

[0024] For at least one corresponding pixel of the corresponding generated thermographic image, a mean value is calculated based on the further image data and the number for averaging, so that the color of the at least one corresponding pixel is determined by the calculated mean value.

[0025] Thus, corresponding temperature values ​​of a certain number of thermographic images are averaged, in particular for corresponding pixels, wherein the corresponding temperature values ​​are represented by colors in the image data. For a number for averaging, which is, for example, 2, two temperature values ​​of corresponding pixels of two corresponding thermographic images are averaged, in particular, in order to obtain a temperature value and thus a corresponding color for the corresponding pixel in the generated thermographic image.

[0026] Furthermore, it is optionally provided that the calculation of the average value is performed only for at least one determined partial area of ​​the generated thermal imaging image. Thus, for example, the image can be cut into three strips (Balke), and averaging is performed only in the middle strip. Alternatively, it can be cut into an inner zone and an outer zone, and averaging is performed only in the inner zone. It is also conceivable that based on the analysis of the image data, for example, the approximate area of ​​the object to be detected is obtained by a detection algorithm, so that averaging is then performed only in the obtained approximate area of ​​the object to be detected. By reducing the averaging to at least one determined partial area, the computational workload can be advantageously reduced.

[0027] In another possibility, it can be provided that at least providing, in particular providing image data and / or determining, in particular determining temperature differences, is performed periodically or based on a trigger condition in order to determine a new quantity for averaging for evaluation. It is therefore conceivable that providing image data and / or determining temperature differences are performed at time intervals of, for example, 10 seconds or after a defined number of thermal imaging images detected by at least one sensor. It can also be provided that the trigger condition is the operation of a button by the user. In this case, at least one sensor can be a thermal imaging camera sensor including, for example, an infrared detector array and can be part of a camera system according to the invention, which additionally includes a button for initiating the provision of image data and / or the determination of temperature differences.

[0028] Likewise, the subject of the invention is also a computer program, in particular a computer program product, which contains instructions which, when executed by a computer, cause the computer to carry out the method according to the invention. Thus, the computer program according to the invention brings the same advantages as those described in detail with reference to the method according to the invention.

[0029] Likewise, the subject matter of the invention is also a device for data processing, which is configured to implement the method according to the invention. For example, a computer that implements the computer program according to the invention can be provided as the device. The computer can have at least one processor for implementing the computer program. It is also possible to provide a non-volatile data memory in which the computer program can be stored and from which the processor can read the computer program for implementation.

[0030] Likewise, the subject of the present invention may also be a computer-readable storage medium having a computer program according to the present invention and / or comprising instructions which, when implemented by a computer, cause the computer to implement the method according to the present invention. The storage medium is, for example, constructed as a data storage such as a hard disk and / or a non-volatile memory and / or a memory card. The storage medium may, for example, be integrated into a computer.

[0031] Likewise, the subject matter of the invention may also be a camera system comprising at least one sensor, in particular a thermal imaging camera sensor, a device according to the invention for data processing, and a display, wherein the at least one sensor is designed to detect image data and further image data. The camera system according to the invention thus brings the same advantages as those described in detail with reference to the method according to the invention.

[0032] Besides this, the method according to the invention can also be implemented as a computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Further advantages, features and details of the present invention are apparent from the following description, in which embodiments of the present invention are described in detail with reference to the accompanying drawings. The features mentioned within the scope of the present invention may be important to the invention individually or in any combination. The accompanying drawings show:

[0034] Figure 1 Schematic visualization of a method, a camera system with a sensor, an apparatus, a storage medium and a computer program according to embodiments of the invention,

[0035] Figure 2 A schematic diagram of a camera system according to an embodiment of the present invention in a perspective front view,

[0036] Figure 3 A schematic diagram of a camera system according to an embodiment of the invention in a perspective rear view,

[0037] Figure 4 A schematic diagram of a method for determining a quantity for averaging according to an embodiment of the present invention,

[0038] Figure 5 A schematic illustration of a method for evaluating further image data according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0039] exist Figure 1 In the figure, a method 100, a camera system 1 with a sensor 2, an apparatus 10, a storage medium 15 and a computer program 20 according to an embodiment of the present invention are schematically shown.

[0040] Figure 1 In particular, an exemplary embodiment of a method 100 for reducing noise in image data is shown. In a first step 101, image data are provided, wherein the image data include at least one thermographic image, wherein the image data are generated by detection by at least one sensor 2. In a second step 102, temperature differences present in the at least one thermographic image are ascertained based on an evaluation of temperature data of the at least one thermographic image. In a third step 103, a number for averaging is determined based on the ascertained temperature differences to evaluate further image data, wherein the number for averaging predetermines how many thermographic images are respectively averaged to evaluate the further image data, wherein the further image data are generated by further detection by at least one sensor 2. In a fourth step, noise in the further image data is reduced by evaluation using the determined number for averaging.

[0041] The camera system 1 according to the invention is described below. Figure 2 and Figure 3 An exemplary embodiment of the camera system 1 is shown in a perspective front view or in a perspective rear view. The camera system 1 is designed to determine two-dimensional temperature data, in particular a thermal image, of a scene to be investigated. The scene can be any arrangement to be investigated, which typically includes objects, in particular surfaces of objects or other similar objects. Examples of such scenes can be house facades, safes, groups of people, landscapes or other similar scenes.

[0042] The camera system 1 comprises a housing 16 with a handle 18. The user can hold the camera system 1 in his hand using the handle 18. In addition, the housing 16 of the camera system 1 has an output device in the form of a touch-sensitive screen 22 and an operating element 24 for user input and control of the camera system 1 on the side 25 facing the user during use of the camera system 1. The camera system 1 also has a button 24a, which the user can use to start the acquisition of two-dimensional temperature data of the scene to be investigated. An access opening 28 is defined in the housing 16 on the side 26 of the housing 16 facing away from the user. The access opening 28 defines the detection range of the camera system 1 (if necessary in conjunction with a lens of the camera system 1, not shown here). The infrared radiation emitted in the solid angle range or in the solid angle range of the scene, in particular in the solid angle range of objects in the scene, is detected by the camera system 1. Directly behind the access opening 28, a lens system is present as a lens in a light tube 32 that reduces stray light (not shown here in detail). The lens system is transparent for radiation in the middle infrared range and serves to focus the infrared radiation onto the infrared detector array of the camera system 1 .

[0043] The projection device 34 is located in the housing 16 on the side 26 of the housing 16 facing away from the user during the use of the camera system 1, and is configured to convert the two-dimensional temperature data obtained by means of the infrared detector array into a projectable image and project the projectable image onto the scene. In the described embodiment, the projection device 34 includes an evaluation device (not shown in detail) and a video projector 34a ("Beamer") arranged in the housing 16 of the camera system 1. The image 36 projected onto the scene 14 is used to enhance the environment defined by the scene, in particular by the projection surface of the scene.

[0044] Furthermore, the camera system 1 can also include a camera operating in the visible spectrum (not shown further here) for recording visible images. Such images can be output together with the thermal image generated based on the temperature measurement initiated by the user, in particular at least partially superimposed or blended with the thermal image.

[0045] Furthermore, the handle 18 has, on the underside of the thermal imaging camera 10 , a receptacle 40 for accommodating an energy storage device 42 , which can be embodied, for example, in the form of a rechargeable accumulator or in the form of a battery.

[0046] Figure 4 An exemplary embodiment of a method for determining a quantity for averaging for evaluating image data is shown. In a first step 201, a thermographic image with temperature data is provided. In a second step 202, temperature differences are determined in the provided thermographic image. In a third step 203, the temperature differences are evaluated. In the case of a high value of, for example, 50° C., or exceeding a correspondingly defined threshold value, a value of 1 is selected for the quantity for averaging according to step 204a. In the case of an intermediate value of, for example, 10° C., or being located in a correspondingly defined range, a value of 2 is selected for the quantity for averaging according to step 204b. In the case of a low value of, for example, 5° C., or being below a correspondingly defined threshold value, a value of 4 is selected for the quantity for averaging according to step 204c. Subsequently, in accordance with step 205, the correspondingly selected quantity for averaging is determined or specified for evaluating further image data.

[0047] Figure 5 An exemplary embodiment of a method for evaluating further image data is shown. In this case, in a first step 301, further image data are provided. In a second step 302, an average value is determined pixel-wise based on the determined number for averaging. Subsequently, in a third step 303, the thermographic image resulting from the pixel-wise calculation is displayed, in particular on the screen 22 of the camera system 1.

[0048] The image quality of a thermal imaging camera is determined in particular by the resolution and the noise (NETD). The NETD describes the “Noise Equivalent Temperature Difference”, ie the thermal noise in the image. In this case, a NETD of 50 mK corresponds, for example, to the standard deviation of the image noise in ° C.

[0049] In the thermographic image, the temperature is preferably presented in color, wherein the temperature measured in the scene is typically displayed in 256 or 512 colors. Typically, the minimum span can be 5°C, wherein the span is the difference between the minimum temperature and the maximum temperature displayed in the image. In the case of a small temperature difference of, for example, 1°C, the span is, for example, 5°C, which can be displayed in 512 colors and can result in a display of approximately 0.01°C per color. This corresponds, for example, to 5 colors per NETD. A large temperature difference of, for example, 100°C results, for example, in a span of 100°C, which can be displayed in 512 colors and can result in a display of approximately 0.2°C per color. This corresponds in particular to 1 color per NETD.

[0050] According to an embodiment, the invention shall select a suitable averaging of the individual images according to the span resulting from the temperature differences in the images and thus preferably also establish an optimum between image refresh rate and noise.

[0051] For example, the span resulting from the temperature difference should be calculated. If the temperature threshold of each color is close to or exceeds the noise (for example 0.2°C compared to 50mK NETD), then averaging multiple images in time may result in a small or only relatively small image improvement. Here, a small averaging in time and a correspondingly higher image refresh rate should preferably be selected. Conversely, in the case of a small span, it may be advantageous to select a larger averaging in time at the expense of a smaller image refresh rate. In addition to these two settings, it is also possible to define additional areas. That is, a very small span may result in a very large averaging, a small span may result in a large averaging, an intermediate span may result in an ordinary averaging, a large span may result in a small averaging, and a very large span may result in a very small averaging. A smooth transition from one area to the next can also be envisioned.

[0052] The device 10 or the computing unit for data processing provided in the camera system 1 according to the embodiment preferably continuously determines the temperature difference and the span from the measured temperature data of the corresponding thermographic images. The number of thermographic images used for the temporal averaging is then adjusted according to the embodiment.

[0053] The above explanation of the embodiments only describes the invention within the scope of examples. Of course, the individual features of the embodiments - as long as they are technically meaningful - can be combined freely with one another without departing from the scope of the invention.

Claims

1. A method (100) for reducing noise in image data, comprising the following steps: - providing (101) image data, wherein the image data comprises at least one thermal imaging image, wherein the image data is generated by detection by at least one sensor (2), - determining (102) temperature differences represented in the at least one thermographic image based on an analysis of temperature data of the at least one thermographic image, - determining (103) a number for averaging to evaluate further image data based on the determined temperature differences, wherein the number for averaging predetermines how many thermographic images are respectively averaged to evaluate the further image data, wherein the further image data are generated by further detection of the at least one sensor (2), - reducing (104) noise in the further image data by evaluating using the determined quantity for averaging.

2. The method (100) according to claim 1, characterized in that: Determining (103) the number for averaging comprises the following steps: - defining at least two regions for the temperature difference, wherein the defined quantity for averaging is assigned to the corresponding region, - determining the quantity for averaging based on a comparison of the ascertained temperature difference with at least two regions defined for the temperature difference.

3. The method (100) according to any one of the preceding claims, characterized in that The method further comprises the following steps: The further image data are evaluated, wherein a correspondingly generated thermographic image is ascertained based on the quantity used for averaging.

4. The method (100) according to claim 3, characterized in that: The method further comprises the following steps: - Initiating output of said corresponding generated thermographic image.

5. The method (100) according to claim 3 or 4, characterized in that: The evaluation of the further image data comprises the following steps: - calculating, for at least one corresponding pixel of the corresponding generated thermographic image, an average value based on the further image data and the number for averaging, in order to determine the color of the at least one corresponding pixel by means of the calculated average value.

6. The method (100) according to claim 5, characterized in that: The averaging is performed only for at least one specific partial region of the generated thermographic image.

7. The method (100) according to any one of the preceding claims, characterized in that At least the providing (101) and / or the ascertaining (102) is performed periodically or based on a trigger condition in order to determine a new quantity for averaging for the evaluation.

8. A computer program (20) comprising instructions which, when executed by a computer (10), cause the computer to carry out the method (100) according to one of the preceding claims.

9. A device (10) for data processing, which is configured to carry out the method (100) according to one of claims 1 to 7.

10. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer to implement the method (100) according to any one of claims 1 to 7.

11. A camera system (1) comprising at least one sensor (2), a device (10) for data processing according to claim 9 and a display (3), wherein: The at least one sensor (2) is designed to detect image data and further image data.

Images