Determining temperature of moving object using temperature measurement camera and such temperature measurement camera
Patent Information
- Application Number
- JP2024109342
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-07-20
- Filing Date
- 2024-07-08
- Publication Date
- 2025-02-26
Smart Images
Figure 2025026821000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a temperature measuring camera for determining the temperature of a moving object. Furthermore, the invention also includes a method for determining the temperature of a moving object using a temperature measuring camera. [Background technology]
[0002] In many situations, temperature measuring cameras are excellent devices for automatic detection and recognition monitoring. In recent years, there has been a focus on making temperature measuring cameras for various temperature applications. One application is to monitor vehicles passing through certain critical road sections, such as tunnels and bridges, with the aim of finding overheated brakes, etc. and preventing fire accidents. In doing so, temperature thresholds for hazardous and non-hazardous events can be monitored. However, such monitoring typically requires the vehicle to slow down or even stop completely for accurate measurements. This is because measurements by temperature measuring cameras are typically subject to motion blur when the object is moving. Furthermore, temperature measuring cameras have a finite thermal resolution and need to be exposed to thermal radiation for a certain minimum time to detect the temperature. Summary of the Invention
[0003] In view of the above, it is an object of the present invention to provide a temperature measuring camera and method for determining the temperature of a moving object.
[0004] According to a first aspect, there is provided a method for determining the temperature of a moving object using a temperature measuring camera. The temperature measuring camera is preferably stationary. The method comprises: capturing a sequence of image frames of the object using a bolometer sensor of the temperature measuring camera while the object is moving; identifying an area corresponding to the object in each image frame of the series of image frames of the sequence of image frames; combining the identified areas from each image frame in the series of image frames into a stacked image of the object, where pixel values in the stacked image of the object are estimated as the sum of pixel values of corresponding pixels in the image frames in the series of image frames; and determining the temperature of the moving object from the pixel values in the stacked image of the object.
[0005] Such a method for determining the temperature of a moving object using a temperature measuring camera facilitates the determination of the temperature of the moving object, for example by ensuring that sufficient statistics regarding the thermal radiation of the object are used to arrive at a reliable temperature determination of the object. This method can provide for monitoring of hazardous and non-hazardous events based on the temperature of the object. As an example, the braking of a train can be monitored while the train is in motion, without the need to slow down the train or with much fewer stops.
[0006] The method may further include estimating a pixel velocity of the moving object from an image frame in the sequence of image frames of the object, calculating a pixel exposure time for an image frame of the sequence of image frames from the pixel velocity of the object and the frame integration time, and determining a number of image frames to be used for the series of image frames based on the calculated pixel exposure time and a thermal time constant of the bolometer sensor.
[0007] The method may further include obtaining a thermal time constant of the bolometer sensor as a value provided by a manufacturer of the bolometer sensor and determining it by one or more of: calibrating the temperature measuring camera at installation.
[0008] The method may further comprise selecting a series of image frames from the sequence of images as those image frames depicting an object in a central portion of the field of view of the temperature measuring camera.
[0009] The method estimates the angular distance traveled by a moving object during a total integration time α from image frames in a sequence of images of the object, and reduces the radiance blurring due to motion using a modular transfer function TIFF2025026821000002.tif10170, where f is the spatial frequency, and correcting pixels of the series of image frames based on the estimated blur. The angular distance traveled during the total integration time can be expressed as α=v×t / D, where v is the velocity, t is the integration time, and D is the distance between the object and the camera. D may be scaled by the angle β from the central optical axis, i.e., Dcosβ. This angle β is easily estimated since the field of view is known as a function of pixel position.
[0010] According to a second aspect, a non-transitory computer readable storage medium is provided having stored thereon computer code instructions executable by a device having processing capabilities, the computer code instructions being configured when executed to instruct the device to receive a sequence of image frames of a moving object captured using a bolometer sensor of a stationary thermometric camera, identify in each image frame of the series of image frames of the sequence of images a region corresponding to the object, combine the identified regions from each image frame in the series of image frames into a stacked image of the object having pixel values estimated as a sum of pixel values of corresponding pixels in the image frames in the series of image frames, and determine a temperature of the moving object from the pixel values in the stacked image of the object.
[0011] The computer code instructions, when executed, may be further configured to instruct the apparatus to estimate a pixel velocity of the moving object from an image frame in the sequence of image frames of the object, calculate a pixel exposure time for an image frame of the sequence of image frames from the pixel velocity of the object and the frame integration time, and determine a number of image frames to be used for the series of image frames based on the calculated pixel exposure time and a thermal time constant of the bolometer sensor.
[0012] The above mentioned features of the method also apply to this second aspect, where applicable, and in order to avoid excessive repetition, reference is made to the above.
[0013] According to a third aspect, there is provided a temperature measuring camera for determining a temperature of a moving object, the temperature measuring camera comprising a bolometer sensor configured to capture a sequence of image frames of the object while the object is moving, and a circuit configured to perform an object identification function configured to identify an area corresponding to the object in each image frame of the series of image frames of the sequence of image frames, a combination function configured to combine the identified areas from each image frame in the series of image frames into a stacked image of the object having pixel values estimated as a sum of pixel values of corresponding pixels in image frames in the series of image frames, and a temperature determination function configured to determine a temperature of the moving object from the pixel values in the stacked image of the object.
[0014] The synthesis function may be further configured to estimate a pixel velocity of the moving object based on image data of the image frames in the sequence of image frames of the object, calculate a pixel exposure time for the image frames of the sequence of image frames from the pixel velocity of the object and the frame integration time, and determine a number of image frames to be used for the series of image frames based on the pixel exposure time and the thermal time constant of the bolometer sensor.
[0015] The above mentioned features of the method also apply to this third aspect, where applicable, and in order to avoid undue repetition reference is made to the above.
[0016] Further areas of applicability will become apparent from the detailed description provided hereinafter. However, it should be understood that the detailed description and specific examples are given by way of example only.
[0017] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. It should be noted that, as used in this specification and the appended claims, the articles "a," "an," "the," and "said" are intended to mean that there are one or more elements, unless the context clearly dictates otherwise. Thus, for example, reference to "a unit" or "the unit" may include several devices, etc. Furthermore, "comprising," "including," "containing," and similar expressions do not exclude other elements or steps.
[0018] The above-mentioned aspects and other aspects will now be described in more detail with reference to the accompanying drawings, which should not be considered limiting, but rather are used for explanation and understanding.
[0019] As shown in the figures, sizes of layers and regions may be exaggerated for illustrative purposes and, therefore, are provided to illustrate the overall structure, and like reference numbers refer to like elements throughout. [Brief description of the drawings]
[0020] [Figure 1] 1 shows a temperature measuring camera. [Diagram 2] 2 is a schematic diagram of a sequence of image frames captured by a temperature measuring camera; [Diagram 3] FIG. 1 is a block diagram for determining the temperature of a moving object using a temperature measuring camera. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] The present invention will now be described more fully hereinafter with reference to the accompanying drawings showing presently preferred embodiments of the invention, however, the invention may be embodied in many different forms.
[0022] FIG. 1 shows a temperature measuring camera 100. The temperature measuring camera 100 can determine the temperature of a moving object. The temperature measuring camera 100 is typically configured to be fixedly positioned. By "fixedly positioned" it is meant that the temperature measuring camera 100 is positioned at a fixed location. Furthermore, the temperature measuring camera 100 may be a pan-tilt-zoom camera or a fixed camera without pan and tilt capabilities. Furthermore, the temperature measuring camera 100 typically has a fixed zoom setting for the sequence of image frames being captured of the object.
[0023] The temperature measuring camera comprises a bolometer sensor 110 and a circuit 120. It is understood that the temperature measuring camera 100 may comprise additional components and features such as a power source, a network connection, a display, input means, etc. However, such additional components and features do not contribute to the functionality of the present invention and therefore will not be described in further detail in this disclosure.
[0024] The bolometer sensor 110 is configured to capture a sequence of image frames of an object, in particular, the bolometer sensor 110 is configured to capture a sequence of image frames of the object while the object is moving.
[0025] An example of such a sequence of image frames is shown diagrammatically in Figure 2. The sequence comprises a number of image frames 210a-210j. Schematically shown throughout the image frames 210a-210j is a moving object 220. The object 220 enters the scene depicted in the sequence of image frames from the left, moves through the scene and exits to the right.
[0026] The bolometer sensor 110 comprises a number of pixels. Moreover, the pixels of the bolometer sensor 110 have a finite thermal resolution and need to be exposed to thermal radiation for a certain time to sense the temperature. The finite thermal resolution is usually called the thermal time constant τ of the bolometer sensor 110. The thermal time constant τ depends on the thermal conductance G and the thermal capacitance C of the bolometer sensor 110 according to the following equation: TIFF2025026821000003.tif7170
[0027] The thermal time constant τ is typically specified by the manufacturer of the bolometer sensor and is usually given with a value of 1-1 / e≈63%. It usually has a value of approximately 10-15 milliseconds. Furthermore, the thermal time constant τ varies with the ambient temperature. Therefore, the thermal time constant τ may be given in a look-up table for various ambient temperatures. Such a look-up table is typically given by the manufacturer of the bolometer sensor. Alternatively, the thermal time constant τ may be determined by calibration of the temperature measuring camera at the time of installation of the temperature measuring camera. Such a calibration can be performed, for example, by utilizing a frame integration time. The frame integration time is set relatively short, for example 1 millisecond, and then integrated until the pixel readout signal is equal to the temperature measurement calibration value at a known scene temperature or shutter flag temperature.
[0028] Furthermore, the bolometer sensor 110 is set to capture thermal radiation from the scene during a frame integration time before the pixel values of the pixels are read out. Such a frame integration time is typically on the order of a few tens of milliseconds. According to one example, the bolometer sensor 110 is set to have a frame rate of 30 frames per second. Under such conditions, the frame integration time is about 30 milliseconds. Therefore, under such conditions, the frame integration time is sufficiently above the thermal time constant τ, and the pixels of the bolometer sensor 110 are exposed to thermal radiation for a sufficient time to reliably sense the temperature. However, when the object whose temperature is to be determined / detected is moving, the information is spread over several pixels of the bolometer sensor 110 and becomes smeared. For example, if the object is moving at a speed of 6 pixels per frame, this means that each pixel has only about 1 / 6 of the true signal. In this example, the frame rate is set to 30 frames per second, which means that the frame integration time is about 30 milliseconds. Each pixel is therefore exposed to thermal radiation from a particular part of the object for only 5 milliseconds in this case. Here, the time that a pixel is exposed to thermal radiation from a particular part of an object is called the pixel exposure time. Thus, if the pixel exposure time is less than the thermal time constant τ, the pixel is not exposed to thermal radiation for a sufficient time to obtain adequate statistics for sensing the temperature. A way to compensate for the lack of statistics is to upscale the radiance of that pixel by extrapolation, which allows for a better radiance / temperature estimate.
[0029] However, the inventors of the present invention have realised that using a temperature measuring camera to determine the temperature of a moving object can be done in another way, by identifying an area corresponding to the object in each image frame of a series of image frames and stacking them to create a sufficiently long integration time or compounding. This way of determining the temperature of a moving object using a temperature measuring camera will be explained in more detail below. First, however, some further details regarding the features of the temperature measuring camera 100 are presented.
[0030] As mentioned above, the temperature measuring camera 100 comprises a circuit 120. The circuit 120 is configured to perform the overall control of the functions and operations of the temperature measuring camera 100. The circuit 120 may include a processor 121, such as a central processing unit CPU, a graphic processing unit GPU, a microcontroller, or a microprocessor. The processor 121 is configured to execute program codes stored in a memory 130 to perform the functions and operations of the temperature measuring camera 100.
[0031] The memory 130 may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory RAM, or another suitable memory unit. In a typical configuration, the memory 130 may include a non-volatile memory for long-term data storage and a volatile memory that serves as a system memory for the circuit 120. The memory 130 may exchange data with the circuit 120 via a data bus. Additionally, there may be associated control lines and an address bus between the memory 130 and the circuit 120.
[0032] The functions and operations of the temperature measuring camera 100 may be embodied in the form of executable logic routines (e.g., lines of code, software programs, etc.) stored in a non-transitory computer-readable medium (e.g., memory 130) of the temperature measuring camera 100 and executed by the circuit 120 (e.g., using the processor 121). Furthermore, the functions and operations of the temperature measuring camera 100 may be standalone software applications or may form part of a software application that performs further tasks related to the temperature measuring camera 100. The functions and operations described may be considered as methods that the corresponding parts of the temperature measuring camera are configured to perform. Also, although the functions and operations described may be implemented in software, such functions may be performed by dedicated hardware or firmware, or any combination of hardware, firmware, and / or software.
[0033] As mentioned above, the inventors of the present invention have realised that when a temperature measuring camera is used to determine the temperature of a moving object, the pixel exposure time falling short of the thermal time constant τ can be compensated for by identifying an area corresponding to the object in each image frame of a series of image frames and stacking them to obtain a sufficiently long integration time. To do so, the circuit 120 of the temperature measuring camera 100 is configured to perform several functions.
[0034] The circuit 120 is configured to perform an object identification function 132. The object identification function 132 is configured to identify regions corresponding to moving objects in each image frame of the series of image frames of the sequence of image frames captured by the bolometer sensor 110. The series of image frames is typically a series of consecutive image frames. The series of image frames and the sequence of image frames may be the same image frames. However, the series of image frames may be a subset of the image frames in the sequence of image frames. Standard motion and / or object detectors can be used to identify pixels belonging to regions of different image frames in the series of image frames. The monitored object may be, for example, a wheel of a train or another type of vehicle.
[0035] The circuit 120 is further configured to perform a combining function 134. The combining function 134 is configured to combine the identified regions from each image frame in the series of image frames into a stacked image of the object. Thus, the composite image of the object is formed by summing pixel values of corresponding pixels in the image frames in the series of image frames. Here, corresponding pixels in the image frames in the series of image frames refer to pixels of the same location of the object in different image frames. That is, for example, pixels corresponding to the center of the object are identified in different image frames and then summed. In another example, pixels corresponding to certain other points of the object may be identified in different image frames and then summed.
[0036] The combining function 134 may be configured to estimate a pixel velocity of the moving object based on the image data of the image frames in the sequence of image frames of the object. According to one example, the pixel velocity of the moving object is estimated to be 6 pixels per frame. The combining function 134 may then be further configured to calculate a pixel exposure time from the pixel velocity of the moving object and the frame rate of the bolometer sensor 110. In the above example, the frame rate of the bolometer sensor 110 is set to be 30 frames per second, considering that the frame integration time of each frame is about 30 milliseconds. Thus, the pixel exposure time is calculated to be 30 / 6 milliseconds = 5 milliseconds. The combining function 134 may then be further configured to determine the number of image frames to be used for the series of image frames based on the calculated pixel exposure time and the thermal time constant τ of the bolometer sensor 110. As exemplified above, the thermal time constant τ of the bolometer sensor 110 may be 15 milliseconds. In that case, the determined number of image frames to be used for the series of image frames is 15 / 5 = 3 image frames. This determined number of image frames to be used for the series of image frames is considered to be the minimum number of image frames to be used for the series of image frames, in fact it is possible to use more image frames than this minimum number of image frames.
[0037] A pixel value in the stacked image of the object may be estimated as a weighted sum of pixel values of corresponding pixels in image frames in the series of image frames. The weights used in the weighted sum may vary between 0 and 1. If the number of image frames used to form the stacked image of the object is set to a minimum number of image frames, the weights are preferably set to 1 or very close to 1. However, if more than this minimum number of image frames are used, smaller weights may be used. For example, the weights may vary depending on where the object is located in the camera's field of view. Image frames having an object in a central portion of the field of view may be given a larger weight compared to image frames in which the object has just entered the scene or is about to exit the scene. As an example, returning to the sequence of image frames described in connection with FIG. 2, a relatively large weight may be given to image frames 210d-210f, and a relatively small weight (or zero weight) may be given to image frames 210a-210c and image frames 210g-210j.
[0038] The combining function 134 may further be configured to select image frames from the sequence of image frames that form part of the series of image frames. Such a selection may be made such that image frames depicting an object in a central part of the field of view of the temperature measuring camera 100 are used in the series of image frames. Thus, image frames depicting only a portion of the object and / or image frames in which an object has just entered or is about to leave the scene may be excluded from the selection of image frames used to form the stacked image of the object. By way of example, returning to the sequence of image frames described in connection with FIG. 2, the combining function 134 may select image frames 210d-210f as image frames that constitute the series of image frames.
[0039] The circuit 120 is further configured to perform a temperature determination function 136. The temperature determination function 136 is configured to determine the temperature of a moving object from pixel values in a stacked image of the object. Thus, the object identification function 132 and the combining function 134 are typically set to operate in a radiance / "raw" signal mode on the pixel values, and the temperature determination function 136 then converts the pixel values to a temperature reading.
[0040] The temperature measuring camera 100 may include a transceiver 150, through which the temperature measuring camera 100 is configured to communicate with other devices or services. For example, the transceiver 150 may be configured to transmit temperature readings to one or more other devices and / or services. The transceiver 150 may be configured to transmit / receive data via a data communication network configured to carry data between different devices. Typically, the data is digital data.
[0041] With reference to FIG. 3, a method 300 for determining the temperature of a moving object using a temperature measuring camera is described. The method 300 is preferably executed by a processor. The method is therefore a computer-implemented method. Some or all of the steps of the method 300 may be executed by the temperature measuring camera 100 described above. However, it is equally understood that some or all of the steps of the method 300 may be executed by one or more other devices having similar functionality. The method 300 includes the following steps. The steps may be executed in any suitable order.
[0042] Step S302 of capturing a sequence of image frames of the object while it is moving. The capturing is performed by a bolometer sensor of the temperature measuring camera. Such capturing is discussed in more detail above and to avoid excessive repetition, reference is made to the above discussion.
[0043] In each image frame of the series of image frames of the sequence of image frames, a region corresponding to the object is identified 304. Such identification is discussed in more detail above and to avoid excessive repetition, reference is made to the above discussion.
[0044] Step S306 combines the identified regions from each image frame in the series of image frames into a stacked image of the object. In combining, pixel values in the stacked image of the object are estimated as sums of pixel values of corresponding pixels in image frames in the series of image frames. Such combining is discussed in more detail above, and reference is made to the above discussion to avoid excessive repetition.
[0045] Step S308 of determining the temperature of the moving object from pixel values in the stacked image of the object. Such determination is discussed in more detail above and to avoid excessive repetition, reference is made to the above discussion.
[0046] In other words, the method for determining the temperature of a moving object using a temperature measuring camera can be summarized as follows: 1) Identify the moving pixels and ignore the stationary pixels. 2) Estimate the object's velocity, for example by estimating the pixel velocity. 3) Based on the estimated object's pixel velocity and knowledge of the temperature measuring camera's frame rate, calculate how many frames need to be added together to satisfy the thermal time constant of the temperature measuring camera's bolometer sensor. All calculations are done in radiance / "raw" signal mode and may then be converted back into a temperature reading.
[0047] Those skilled in the art will appreciate that the present invention is in no way limited to what has been expressly described above, but on the contrary, numerous modifications and variations are possible within the scope of the appended claims.
[0048] For example, blurring in image frames captured by a temperature measuring camera due to object motion can be corrected by estimating from an image frame in a sequence of image frames of the object the angular distance traveled by the moving object during a total integration time α and correcting the radiance blurring due to motion using a modular transfer function TIFF2025026821000004.tif10170, where f is the spatial frequency, and by correcting the pixels of a series of image frames based on the estimated blur. The angular distance traveled during the total integration time can be expressed as α=v×t / D, where v is the velocity, t is the integration time, and D is the distance between the object and the camera. D may be scaled by the angle β from the central optical axis, i.e., Dcosβ. This angle β is easily estimated since the field of view is known as a function of pixel position. By correcting for motion blur, object detection can be made more robust. This allows for more reliable temperature determination.
[0049] As alluded to above, there is a limit as to how fast an object can move and still achieve a sufficiently reliable temperature reading. It may be the case that the temperature of an object cannot be determined if the object moves too fast to collect the required number of frames before it moves out of the field of view of the temperature measuring camera. This problem may be solved by using two or more temperature measuring cameras along the trajectory of the object, for example along a railroad track. A first series of image frames may be captured by a first temperature measuring camera as the object passes the first temperature measuring camera, and a second series of image frames may be captured as the object passes the second temperature measuring camera. The first and second series of image frames may then be combined to form a sufficiently long series of image frames that are combined into a stacked image.
[0050] Moreover, those skilled in the art, from a study of the drawings, the disclosure, and the appended claims, can understand and effect variations in practicing the claimed invention.
Claims
1. 1. A method for determining the temperature of a moving object using a fixedly positioned temperature measuring camera, comprising: capturing a sequence of image frames of the object while the object is moving using a bolometer sensor of the temperature measuring camera; identifying a region corresponding to the object in each image frame of a series of image frames in the sequence of image frames; combining the identified regions from each image frame in the series of image frames into a stacked image of the object, where pixel values in the stacked image of the object are estimated as a sum of pixel values of corresponding pixels in the image frames in the series of image frames; determining the temperature of the moving object from pixel values in the stacked image of the object; The method includes:
2. estimating a pixel velocity of the moving object from an image frame in the sequence of image frames of the object; calculating a pixel exposure time for the image frame of the sequence of image frames from the pixel velocity of the object and a frame integration time; determining a number of image frames to be used for the series of image frames based on the calculated pixel exposure time and a thermal time constant of the bolometer sensor; The method of claim 1 further comprising:
3. The thermal time constant of the bolometer sensor is obtaining as a value provided by the manufacturer of the bolometer sensor; and Calibrating the temperature measuring camera upon installation To be determined by one or more of the following: The method of claim 2 , further comprising:
4. selecting said series of image frames from said sequence of images as those image frames depicting said object in a central portion of the field of view of said temperature measuring camera; The method of claim 1 further comprising:
5. estimating from an image frame in the sequence of image frames of the object an angular distance traveled by the moving object during a total integration time α; The motion-induced radiance blur is calculated using the module transfer function where f is the spatial frequency; correcting pixels of the series of image frames based on the estimated blur; and The method of claim 1 further comprising:
6. A non-transitory computer-readable storage medium having stored thereon computer code instructions executable by a device having processing capability, comprising: The computer code instructions, when executed, cause the device to: receiving a sequence of image frames of a moving object captured using a bolometer sensor of a stationary thermometric camera; identifying a region corresponding to the object in each image frame of a series of image frames in the sequence of images; combining the identified regions from each image frame in the series of image frames into a stacked image of the object having pixel values estimated as the sum of pixel values of corresponding pixels in the image frames in the series of image frames; determining a temperature of the moving object from pixel values in the stacked image of the object; 23. A non-transitory computer-readable storage medium configured to:
7. The computer code instructions, when executed, cause the device to: estimating a pixel velocity of the moving object from an image frame in the sequence of image frames of the object; calculating a pixel exposure time for the image frame of the sequence of image frames from the pixel velocity of the object and a frame integration time; determining a number of image frames to be used for the series of image frames based on the calculated pixel exposure time and the thermal time constant of the bolometer sensor; 7. The non-transitory computer-readable storage medium of claim 6, configured to:
8. 1. A temperature measuring camera for determining the temperature of a moving object, comprising: a bolometer sensor configured to capture a sequence of image frames of the object while the object is moving; 1. A circuit comprising: an object identification function configured to identify a region corresponding to the object in each image frame of a series of image frames in the sequence of image frames; a synthesis function configured to synthesize the identified regions from each image frame in the series of image frames into a stacked image of the object having pixel values estimated as a sum of pixel values of corresponding pixels in the image frames in the series of image frames; and a temperature determining function configured to determine the temperature of the moving object from pixel values in the stacked image of the object. and a circuit configured to perform A temperature measuring camera equipped with
9. The synthesis function is estimating a pixel velocity of the moving object based on image data of an image frame in the sequence of image frames of the object; calculating a pixel exposure time for the image frame of the sequence of image frames from the pixel velocity of the object and a frame integration time; determining a number of image frames to be used for the series of image frames based on the pixel exposure time and a thermal time constant of the bolometer sensor; The temperature measuring camera according to claim 8, further configured to: