Temperature compensation in infrared imaging systems and methods

Abstract

Techniques are provided for facilitating temperature compensation. In one example, an infrared imaging system includes a focal plane array configured to capture radiation from a scene and generate image data based on the radiation. The focal plane array also captures radiation from an element associated with the infrared imaging system during the capturing of the radiation from the scene. The infrared imaging system also includes a temperature sensor configured to determine a temperature of the focal plane array. The infrared imaging system also includes processing circuitry configured to determine a temperature associated with the element based on the temperature of the focal plane array. The processing circuitry is also configured to determine a temperature associated with an object in the scene based on the infrared image data, the temperature associated with the element, and the temperature of the focal plane array. Related devices and methods are also provided.

Description

TECHNICAL FIELD

[0001] One or more embodiments relate generally to imaging, and more particularly, for example, to temperature compensation in infrared imaging systems and methods. BACKGROUND

[0002] An imaging system can include an array of detectors, each detector functioning as a pixel to produce a portion of a two-dimensional image. There is a wide variety of image detectors, such as visible light image detectors, infrared image detectors, or other types of image detectors that can be disposed in an image detector array for capturing images. As an example, a plurality of sensors can be disposed in an image detector array to detect electromagnetic (EM) radiation at a desired wavelength. In certain cases, such as for infrared imaging, the readout of image data captured by the detectors can be performed by a readout integrated circuit (ROIC) in a time-multiplexed manner. The readout image data can be communicated to other circuitry, such as for processing, storage, and / or display. In certain cases, the combination of a detector array and a ROIC can be referred to as a focal plane array (FPA). Advances in FPAs and image processing technology have led to increased functionality and complexity of resulting imaging systems. SUMMARY

[0003] In one or more embodiments, an infrared imaging system includes a FPA configured to capture radiation from a scene and generate infrared image data based on the radiation. The FPA also captures radiation from an element associated with the infrared imaging system during the capturing of the radiation from the scene. The infrared imaging system also includes a temperature sensor configured to determine a temperature of the FPA. The infrared imaging system also includes processing circuitry configured to determine a temperature associated with the element based at least in part on the temperature of the FPA. The processing circuitry is also configured to determine a temperature associated with an object in the scene based on the infrared image data, the temperature associated with the element, and the temperature of the FPA.

[0004] In one or more embodiments, a method includes capturing, by a FPA, radiation from an element and from a scene. The method also includes generating, by the FPA, infrared image data based on the radiation. The method also includes determining a temperature of the FPA. The method also includes determining a temperature associated with the element based at least in part on the temperature of the FPA. The method also includes determining a temperature associated with an object in the scene based on the infrared image data, the temperature associated with the element, and the temperature of the FPA.

[0005] The scope of the disclosure is defined by the claims, which are incorporated into this section by reference. Those skilled in the art will obtain a more complete understanding of embodiments of the disclosure, and the advantages thereof, after studying the following detailed description taken in conjunction with the accompanying drawings, which will first be briefly described. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 A block diagram of an example imaging system is shown in accordance with one or more embodiments of the present disclosure.

[0007] Figure 2 A block diagram of an example image sensor assembly is shown in accordance with one or more embodiments of the present disclosure.

[0008] Figure 3 An example system for facilitating temperature compensation is shown in accordance with one or more embodiments of the present disclosure.

[0009] Figure 4 A flow diagram of an example process for facilitating temperature compensation is shown in accordance with one or more embodiments of the present disclosure.

[0010] Embodiments of the present disclosure, and their advantages, are best understood by referring to the following detailed description. It should be noted that the size of various components and the distance between components in the figures are not to scale. It should be understood that like reference numerals are used to identify like elements throughout one or more figures. DETAILED DESCRIPTION

[0011] The detailed description set forth below is intended as a description of various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology can be practiced. The appended drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a thorough understanding of the subject technology. However, it will be clear and apparent to those skilled in the art that the subject technology is not limited to the specific details set forth herein and can be practiced using one or more embodiments. In one or more instances, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technology. One or more embodiments of the present disclosure are illustrated and / or described in relation to one or more figures and / or in conjunction with the same, and are set forth in the claims.

[0012] Various techniques are provided to facilitate temperature compensation in infrared imaging systems and methods. An infrared imaging system (e.g., a thermal camera) can be used to capture infrared image data associated with a scene using an image sensor device (e.g., a detector array of an FPA). The image sensor device includes detectors (e.g., also referred to as detector pixels, detector elements, or simply pixels). Each detector pixel can detect incident EM radiation and generate infrared image data indicative of the detected EM radiation of the scene. In some embodiments, the image sensor array is used to detect infrared radiation (e.g., thermal infrared radiation). For a pixel of an infrared image (e.g., a thermal infrared image), each output value of the pixel can be represented / provided as and / or correspond to a temperature, a digital count value, a percentage of a full temperature range, or generally any value that can be mapped to a temperature. For example, a digital count value of 13000 for a pixel output can represent a temperature of 160°C. Thus, the captured infrared image data can be indicative of or can be used to determine temperatures of objects, people, and / or other features / aspects in the scene.

[0013] In certain instances, the infrared imaging system can present the infrared image data in an image according to a palette. The palette can provide a mapping from different output levels to visual representation values. The palette can be applied to image data values output by the image sensor device of the infrared imaging system (e.g., based on detected EM radiation) to generate an image. In this regard, the image can be considered a visual representation of the image data values. The visual representation values of the palette can include color values and / or grayscale values. In certain instances, the visual representation values can facilitate analysis of the scene by a user of the infrared imaging system and / or circuitry of the infrared imaging system and / or other machines (e.g., machine vision).

[0014] Spatial gradients in the infrared imaging system can be determined (e.g., estimated) by measuring temporal gradients (e.g., rates of temperature change). An infrared imaging system that is externally heated (e.g., when the infrared imaging system is removed from the outside of cold air to the inside of room temperature air) can exhibit a spatial thermal gradient that is related (e.g., strongly related) to the rate of temperature change. In this case, a change in the ambient temperature can change the temperature of the image sensor device of the infrared imaging system relative to the temperature of another component of the infrared imaging system (e.g., a housing).

[0015] During startup (e.g., its initial power-up) or other internal heating processes of an infrared imaging system, the infrared imaging system heats up internally. Then, for the same measured rate of change, the spatial thermal gradient can be different. As one example, startup can end after about five minutes (e.g., steady state begins about five minutes after startup begins). After startup ends, the temperatures and temperature rate of change associated with the various components of the infrared imaging system can generally be due to external factors, rather than internal heating of the infrared imaging system. As an example of an internal heating process other than startup, an internal heating process can be caused by a change in operating mode of the infrared imaging system (e.g., a change in frame rate associated with the FPA that can cause a change in internal heating). As another example, an internal heating process can be caused by turning on / off or otherwise changing power dissipating electronics (e.g., electronics proximate to the FPA). For example, an internal heating process can be caused by changing image processing on a processor that is thermally coupled (e.g., closely thermally coupled) to the FPA.

[0016] In some embodiments, the infrared imaging system can perform temperature compensation to provide temperature measurements with higher accuracy (e.g., higher radiometric accuracy) relative to if compensation were not performed, even during startup modes in which the infrared imaging system exhibits significant internal heating (e.g., due to power-up of various components of the infrared imaging system). In this regard, the compensation can be adjusted between compensation when the infrared imaging system exhibits startup behavior in which internal heating is significant, and compensation when the infrared imaging system exhibits steady state behavior in which temperature changes are primarily due to external factors. In an aspect, the infrared imaging system can be referred to as operating in startup modes / conditions (e.g., also referred to as transient modes / conditions) and steady state modes / conditions (e.g., also referred to as equilibrium modes / conditions). The amount of time the infrared imaging system spends operating in the startup and steady state modes can be based in part on the ambient temperature. For example, if the infrared imaging system is turned on in a cooler ambient temperature, the infrared imaging system can operate in the startup mode for a longer time relative to being turned on in a warmer ambient temperature. In certain cases, the compensation can allow for the gradient due to external temperature changes to be considered differently than the internal temperature changes expected at startup.

[0017] Compensation can account for internal radiation on an image sensor device of an infrared imaging system. Such internal radiation can be considered and referred to as non-scene related radiation / undesired radiation, which is generated by internal components of the infrared imaging system and captured by the image sensor device during operation of the image sensor device to capture scene information. Such internal radiation radiating onto the image sensor device can affect image data of a scene captured by the image sensor device, and thus affect temperatures related to the scene (e.g., temperatures of objects and / or other features / aspects of the scene) determined by the infrared imaging system. As one example, the infrared imaging system can need to compensate for off scene radiation to provide accurate radiometric information.

[0018] In this regard, the infrared imaging system can perform temperature compensation to account for radiation from one or more components (e.g., internal components) of the infrared imaging system that can be received by the image sensor device. As non-limiting examples, components that can radiate onto the image sensor array can include a housing of the infrared imaging system, optics (e.g., lenses, mirrors, etc.) of the infrared imaging system, and / or a shutter of the infrared imaging system. Compensation can be based on temperature data from one or more temperature sensors (e.g., also referred to as thermal sensors, such as thermistors and / or thermocouples). In certain instances, each temperature sensor is located within the housing of the infrared imaging system. Each temperature sensor can be used to measure a temperature of a component of the infrared imaging system that can radiate onto the image sensor array. A thermal gradient (e.g., rate of temperature change) associated with the component can be determined based on measurements of the temperature of the component by the temperature sensor at different points in time.

[0019] For other components that can radiate onto the image sensor array, temperatures of the other components can be determined (e.g., modeled and estimated) based on temperature data from the temperature sensors. In an aspect, as further described herein, the temperatures of the components can be determined using a model (e.g., thermal model). In certain instances, such a model can account for thermal gradients generated by varying external temperatures. Temperature data (e.g., temperature measurements) from one or more thermal sensors can be used as inputs to the model. These models can account for / distinguish between operation in a start-up mode and a steady state mode.

[0020] As one example, the infrared imaging system can be a small camera module that includes a single temperature sensor for measuring the temperature of the FPA. The temperature of one or more other components of the infrared imaging system (e.g., the housing, shutter, and / or lens) can be determined based at least on the temperature measurement of the FPA by the single temperature sensor. As another example, the infrared imaging system can include a temperature sensor for measuring the temperature of the FPA and a temperature sensor for measuring the temperature of the lens. The temperature of other components (e.g., the housing and / or shutter) can be determined based at least on the temperature measurements of the FPA and the lens. In certain instances, it can not be possible to place a temperature sensor on each component itself due to space considerations (e.g., limited space around the components and / or within the housing, placement of a temperature sensor on a component can block the optical path to the FPA, etc.), manufacturing and / or component cost, power considerations (e.g., each placed temperature sensor requires power to operate), and / or other considerations.

[0021] Thus, using various embodiments, external temperature variations can be compensated for / considered through measurement and modeling of internal temperatures of the infrared imaging system. In some aspects, compensation can be performed without using / placing temperature sensors external to the infrared imaging system (e.g., without using an ambient temperature sensor).

[0022] While various embodiments for temperature compensation are described primarily with respect to infrared imaging, temperature compensation using the methods and systems disclosed herein can be used in conjunction with various devices and systems, such as infrared imaging systems, imaging systems with visible and infrared imaging capabilities, short wave infrared (SWIR) imaging systems, light detection and ranging (LIDAR) imaging systems, radar detection and ranging (RADAR) imaging systems, millimeter wave (MMW) imaging systems, ultrasound imaging systems, X-ray imaging systems, microscope systems, mobile digital cameras, video surveillance systems, video processing systems, or other systems or devices that can require obtaining image data at one or more portions of the EM spectrum. For example, the temperature compensated image can be an infrared image of a scene that can be fused / blended with a visible light image of the scene.

[0023] Referring now to the drawings, Figure 1 A block diagram of an example imaging system 100 in accordance with one or more embodiments of the disclosure is shown. However, not all of the depicted components can be required, and one or more embodiments can include additional components not shown in the figures. Changes in the arrangement of components and their functions can be made without departing from the spirit or scope of the claims as set forth herein. Additional components, different components, and / or fewer components can be provided.

[0024] According to embodiments of the present disclosure, the imaging system 100 can be used to capture and process images. The imaging system 100 can represent any type of imaging system that detects one or more ranges (e.g., wavebands) of EM radiation and provides representative data (e.g., one or more still image frames or video image frames). The imaging system 100 can include an imaging device 105. As non-limiting examples, the imaging device 105 can be or can include an infrared camera, a visible light camera, a tablet computer, a laptop computer, a personal digital assistant (PDA), a mobile device, a desktop computer, or other electronic device or can be part of an infrared camera, a visible light camera, a tablet computer, a laptop computer, a personal digital assistant (PDA), a mobile device, a desktop computer, or other electronic device. The imaging device 105 can include a housing that at least partially encloses components of the imaging device 105 to facilitate compactness and protection of the imaging device 105. For example, Figure 1 The solid-line box labeled 105 can represent the housing of the imaging device 105. The housing can contain more, fewer, and / or different components of the imaging device 105 than those depicted within the solid-line box in Figure 1 The solid-line box labeled 105 can represent the housing of the imaging device 105. The housing can contain more, fewer, and / or different components of the imaging device 105 than those depicted within the solid-line box in

[0025] According to one implementation, the imaging system 105 includes a processing component 110, a memory component 115, an image capture component 120, an image interface 125, a control component 130, a display component 135, a sensing component 140, and / or a network interface 145. According to various embodiments, the processing component 110 includes one or more of a processor, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a single-core processor, a multi-core processor, a microcontroller, a programmable logic device (PLD) such as a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) device, or other logic devices that can be configured to perform various operations discussed herein for embodiments of the present disclosure, either by hardwiring, executing software instructions, or a combination of the two. The processing component 110 can be configured to interface and communicate with various other components of the imaging system 100 (e.g., 115, 120, 125, 130, 135, 140, 145, etc.) to perform such operations. For example, the processing component 110 can be configured to process captured image data received from the image capture component 120, store image data in the memory component 115, and / or retrieve stored image data from the memory component 115. In an aspect, the processing component 110 can be configured to perform various system control operations (e.g., controlling communication and operation of various components of the imaging system 100) and other image processing operations (e.g., data conversion, data transformation, data compression, video analysis, etc.).

[0026] In one embodiment, the memory component 115 includes one or more memory devices configured to store data and information, including infrared image data and information. The memory component 115 can include one or more different types of memory devices, including volatile and non-volatile memory devices, such as random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), non-volatile random access memory (NVRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, a hard disk drive, and / or other types of memory. As described above, the processing component 110 can be configured to execute software instructions stored in the memory component 115 to perform method and process steps and / or operations. The processing component 110 and / or the image interface 125 can be configured to store images or digital image data captured by the image capture component 120 in the memory component 115.

[0027] In some embodiments, a separate machine-readable medium 150 (e.g., a memory such as a hard drive, compact disk, digital video disk, or flash memory) can store software instructions and / or configuration data that can be executed or accessed by a computer (e.g., a logic-based device or processor-based system) to perform various methods and operations, such as methods and operations associated with processing image data. In one aspect, the machine-readable medium 150 can be portable and / or located separate from the imaging device 105, where the stored software instructions and / or data are provided to the imaging device 105 by coupling the machine-readable medium 150 to the imaging device 105 and / or by downloading the software instructions and / or data from the machine-readable medium 150 by the imaging device 105 (e.g., via a wired link and / or a wireless link). It should be understood that various modules can be integrated in software and / or hardware as part of the processing component 110, where the code (e.g., software or configuration data) for the modules is stored, for example, in the memory component 115.

[0028] The imaging device 105 can be a video and / or still camera for capturing and processing images and / or video of a scene 175. In this regard, the image capture component 120 of the imaging device 105 can be configured to capture images (e.g., still and / or video images) of the scene 175 in a particular frequency spectrum or modality. The image capture component 120 includes an image detector circuit 165 (e.g., a visible light detector circuit, a thermal infrared detector circuit) and a readout circuit 170 (e.g., a ROIC). For example, the image capture component 120 can include an IR imaging sensor (e.g., an IR imaging sensor array) configured to detect IR radiation in the near, mid, and / or long IR spectrum and provide an IR image (e.g., IR image data or signal) representative of the IR radiation from the scene 175. For example, the image detector circuit 165 can capture (e.g., detect, sense) IR radiation having a wavelength in a range from about 700 nm to about 2 mm, or a portion thereof. For example, in some aspects, the image detector circuit 165 can be sensitive (e.g., better detect) to SWIR radiation, mid-wave IR (MWIR) radiation (e.g., EM radiation having a wavelength of 2 pm to 5 pm), and / or long-wave IR (LWIR) radiation (e.g., EM radiation having a wavelength of 7 pm to 14 pm), or any desired IR wavelength (e.g., generally in the range of 0.7 pm to 14 pm). In other aspects, the image detector circuit 165 can capture radiation from one or more other bands of the EM spectrum, such as visible light, ultraviolet light, etc.

[0029] Image detector circuit 165 can capture image data (e.g., infrared image data) associated with scene 175. To capture a detector output image, image detector circuit 165 can detect image data of scene 175 (e.g., in the form of EM radiation) and generate pixel values of an image based on scene 175. An image can be referred to as a frame or an image frame. In some cases, image detector circuit 165 can include a detector array (e.g., also referred to as a pixel array) that can detect radiation of a particular waveband, convert the detected radiation into an electrical signal (e.g., a voltage, a current, etc.), and generate pixel values based on the electrical signal. Each detector in the array can capture a respective portion of the image data and generate a pixel value based on the respective portion captured by the detector. The pixel value generated by a detector can be referred to as an output of the detector. As non-limiting examples, each detector can be a photodetector, such as an avalanche photodiode, an infrared photodetector, a quantum well infrared photodetector, a microbolometer, or other detector capable of converting EM radiation (e.g., EM radiation of a particular wavelength) into a pixel value. The detector array can be arranged in rows and columns.

[0030] A detector output image can be or can be considered a data structure that includes pixels, and is a representation of image data associated with scene 175, where each pixel has a pixel value that represents EM radiation emitted or reflected from a portion of scene 175 and received by a detector that generates the pixel value. Based on context, a pixel can refer to a detector of image detector circuit 165 that generates an associated pixel value or a pixel (e.g., a pixel location, a pixel coordinate) of a detector output image formed from generated pixel values. In one example, a detector output image can be an infrared image (e.g., a thermal infrared image). For a thermal infrared image (e.g., also referred to as a thermal image), each pixel value of the thermal infrared image can represent a temperature of a corresponding portion of scene 175. In another example, a detector output image can be a visible light image.

[0031] In an aspect, pixel values generated by the image detector circuit 165 can be represented in terms of digital count values generated from electrical signals obtained based on detected radiation from a conversion. For example, where the image detector circuit 165 includes or is otherwise coupled to an analog-to-digital conversion (ADC) circuit, the ADC circuit can generate digital count values based on the electrical signals. For an ADC circuit that can represent electrical signals using 14 bits, the range of digital count values can be from 0 to 16383. In such a case, the pixel values of the detector can be the digital count values output from the ADC circuit. In other cases (e.g., where there is no ADC circuit), the pixel values can be analog in nature, with values that are or indicate values of the electrical signals. As an example, for infrared imaging, a greater amount of IR radiation incident to and detected by the image detector circuit 165 (e.g., an IR image detector circuit) is associated with a higher digital count value and a higher temperature.

[0032] The readout circuit 170 can serve as an interface between the image detector circuit 165 that detects image data and the processing component 110 that processes the detected image data read out by the readout circuit 170, with the image interface 125 facilitating the transfer of data from the readout circuit 170 to the processing component 110. An image capture frame rate can refer to the rate at which the image detector circuit 165 detects / outputs images in sequence (e.g., output images per second by the detector) and provides the images to the processing component 110 by the readout circuit 170. The readout circuit 170 can read out pixel values generated by the image detector circuit 165 according to an integration time (e.g., also referred to as an integration period).

[0033] In various embodiments, the combination of the image detector circuit 165 and the readout circuit 170 can be, can include, or can together provide an FPA. In some aspects, the image detector circuit 165 can be a thermal image detector circuit that includes a microbolometer array, and the combination of the image detector circuit 165 and the readout circuit 170 can be referred to as a microbolometer FPA. In certain cases, the microbolometer array can be arranged in rows and columns. A microbolometer can detect IR radiation and generate a pixel value based on the detected IR radiation. For example, in certain cases, a microbolometer can be a thermal IR detector that detects IR radiation in the form of thermal energy and generates a pixel value based on the amount of thermal energy detected. The microbolometer can absorb incident infrared radiation and produce a corresponding temperature change in the microbolometer. The temperature change is associated with a corresponding change in electrical resistance of the microbolometer. By converting the electrical resistance change of each microbolometer to a time-division multiplexed electrical signal, a two-dimensional image or pictorial representation of the incident IR radiation can be generated, with each microbolometer serving as a pixel. The conversion can be performed by a ROIC. The microbolometer FPA can include IR detection material, such as amorphous silicon (a-Si), vanadium oxide (VOx ), combinations thereof, and / or other detection materials. In an aspect, for a microbolometer FPA, the integration time can be or can indicate a time interval during which the microbolometer is biased. In such cases, a longer integration time can be associated with a higher IR signal gain, but not collect more IR radiation. The IR radiation can be collected by the microbolometer in the form of thermal energy.

[0034] In some cases, the image capture component 120 can include one or more filters adapted to pass some wavelengths of radiation but substantially block other wavelengths of radiation. For example, the image capture component 120 can be an IR imaging device that includes one or more filters adapted to pass certain wavelengths of IR radiation while substantially blocking other wavelengths of IR radiation (e.g., MWIR filters, thermal IR filters, and narrowband filters). In this example, such filters can be used to customize the image capture component 120 to increase sensitivity to IR wavelengths in a desired waveband. In an aspect, when the IR imaging device is customized for capturing thermal IR images, the IR imaging device can be referred to as a thermal imaging device. Other imaging devices, including IR imaging devices that are customized for capturing infrared IR images outside of the thermal range, can be referred to as non-thermal imaging devices.

[0035] In one particular non-limiting example, the image capture component 120 can include an IR imaging sensor having an FPA of detectors responsive to IR radiation including near-infrared (NIR), SWIR, MWIR, LWIR, and / or very long wave infrared (VLWIR) radiation. In some other embodiments, alternatively or additionally, the image capture component 120 can include a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor that can be found in any consumer camera (e.g., a visible light camera).

[0036] Other imaging sensors that can be included in the image capture component 120 include a photonic mixer device (PMD) imaging sensor or other time-of-flight (ToF) imaging sensor, a LIDAR imaging device, a RADAR imaging device, a millimeter imaging device, a positron emission tomography (PET) scanner, a single photon emission computed tomography (SPECT) scanner, an ultrasound imaging device, or other imaging device that operates in a particular modality and / or spectrum. It should be noted that for some of these imaging sensors that are configured to capture images in a particular modality and / or spectrum (e.g., infrared spectrum, etc.), they are more prone to produce images with low-frequency shading, for example, when compared to a general CMOS- or CCD-based imaging sensor or other imaging sensor, imaging scanner, or imaging device of a different modality.

[0037] The image provided by the image capture unit 120, or the corresponding digital image data, can be associated with a corresponding image size (also known as pixel size). Image size or pixel size typically refers to the number of pixels in an image; for example, for a two-dimensional image, it can be represented as width multiplied by height, or otherwise in a way suitable for the image's relevant dimensions or shape. Therefore, the size of an image with its original resolution can be resized to be smaller (e.g., with a smaller pixel size) to, for example, reduce the cost of processing and analyzing the image. Filters (e.g., non-uniformity estimates) can be generated based on the analysis of the resized image. The size of the filter can then be adjusted to the original resolution and size of the image before being applied to it.

[0038] In some embodiments, the image interface 125 may include suitable input ports, connectors, switches, and / or circuitry configured to interface with external devices (e.g., remote device 155 and / or other devices) to receive images (e.g., digital image data) generated by or otherwise stored at the external device. In one aspect, the image interface 125 may include a serial interface and telemetry lines for providing metadata associated with the image data. The received image or image data may be provided to the processing unit 110. In this regard, the received image or image data may be converted into signals or data suitable for processing by the processing unit 110. For example, in one embodiment, the image interface 125 may be configured to receive analog video data and convert it into suitable digital data for provision to the processing unit 110.

[0039] Image interface 125 may include various standard video ports that can be connected to a video player, camera, or other device capable of generating standard video signals, and can convert received video signals into digital video / image data suitable for processing by processing unit 110. In some embodiments, image interface 125 may also be configured to connect to and receive images (e.g., image data) from image capture unit 120. In other embodiments, image capture unit 120 may be directly interfaced with processing unit 110.

[0040] In one embodiment, the control component 130 includes user input and / or interface devices, such as a rotatable knob (e.g., a potentiometer), buttons, a slide bar, a keyboard, and / or other devices, which are adapted to generate user input control signals. The processing component 110 can be configured to sense control input signals from a user via the control component 130 and respond to any sensed control input signals received therefrom. As is generally understood by those skilled in the art, the processing component 110 can be configured to interpret such control input signals as values. In one embodiment, the control component 130 can include a control unit (e.g., a wired or wireless handheld control unit) having buttons adapted to interact with a user and receive user input control values. In one implementation, the buttons of the control unit can be used to control various functions of the imaging device 105, such as auto focus, menu enablement and selection, field of view, brightness, contrast, noise filtering, image enhancement, and / or various other functions.

[0041] In one embodiment, the display component 135 includes an image display device (e.g., a liquid crystal display (LCD)) or various other types of generally known video displays or monitors. The processing component 110 can be configured to display image data and information on the display component 135. The processing component 110 can be configured to retrieve image data and information from the memory component 115 and display any retrieved image data and information on the display component 135. The display component 135 can include display circuitry, which can be used by the processing component 110 to display image data and information. The display component 135 can be adapted to receive image data and information directly from the image capture component 120, the processing component 110, and / or the image interface 125, or can transfer image data and information from the memory component 115 via the processing component 110. In some aspects, the control component 130 can be implemented as part of the display component 135. For example, a touch screen of the imaging device 105 can provide both the control component 130 (e.g., for receiving user input via taps and / or other gestures) and the display component 135 of the imaging device 105.

[0042] In one embodiment, the sensing component 140 includes one or more sensors of different types, as will be appreciated by those skilled in the art, depending on the application or implementation requirements. The sensors of the sensing component 140 provide data and / or information to the processing component 110, at least. In one aspect, the processing component 110 can be configured to communicate with the sensing component 140. In various implementations, the sensing component 140 can provide information regarding environmental conditions, such as outdoor temperature, lighting conditions (e.g., day, night, dusk, and / or dawn), humidity levels, specific weather conditions (e.g., sun, rain, and / or snow), distance (e.g., laser rangefinder or time-of-flight camera), and / or whether a tunnel or other type of enclosure has been entered or exited. The sensing component 140 can represent conventional sensors that are generally known to those skilled in the art for monitoring various conditions (e.g., environmental conditions) that can have an impact (e.g., on the appearance of an image) on the image data provided by the image capture component 120.

[0043] In some implementations, the sensing component 140 (e.g., one or more sensors) can include devices that relay information to the processing component 110 via wired and / or wireless communication. For example, the sensing component 140 can be adapted to receive information from satellites, through local broadcast (e.g., radio frequency (RF)) transmissions, through mobile or cellular networks, and / or through information beacons in infrastructure (e.g., traffic or highway information beacon infrastructure) or various other wired and / or wireless technologies. In some embodiments, the processing component 110 can use information (e.g., sensing data) acquired from the sensing component 140 to modify the configuration of the image capture component 120 (e.g., adjust light sensitivity levels, adjust the direction or angle of the image capture component 120, adjust the aperture, etc.).

[0044] In some embodiments, the sensing component 140 can include a temperature sensing component to provide temperature data (e.g., one or more measured temperature values) associated with the scene 175. The temperature sensing component can include one or more temperature sensors. In some cases, the temperature sensors can be non-contact temperature sensors or contact temperature sensors. By way of non-limiting example, the temperature sensors can include thermistors, thermocouples, thermopiles, pyrometers, and / or other suitable sensors for providing temperature data. In some cases, the temperature data captured by the temperature sensing component can be provided to the processing component 110 through a temperature data interface. The temperature data interface can receive the temperature data and convert the temperature data to a format suitable for the processing component 110.

[0045] In some embodiments, the various components of the imaging system 100 can be distributed over the network 160 and in communication with each other via the network 160. In this regard, the imaging device 105 can include a network interface 145 configured to facilitate wired and / or wireless communication between the various components of the imaging system 100 over the network 160. In such embodiments, components can also be replicated if required for a particular application of the imaging system 100. That is, components configured for the same or similar operations can be distributed over the network. Moreover, all or a portion of any of the various components can be implemented using appropriate components of a remote device 155 (e.g., a conventional digital video recorder (DVR), a computer configured for image processing, and / or other device) in communication with the various components of the imaging system 100 via the network interface 145 over the network 160 if desired. Thus, for example, all or a portion of the processing component 110, all or a portion of the memory component 115, and / or all or a portion of the display component 135 can be implemented at or replicated in the remote device 155. In some embodiments, the imaging system 100 can not include an imaging sensor (e.g., the image capture component 120) but rather receive images or image data from an imaging sensor that is located separate and apart from the processing component 110 and / or other components of the imaging system 100. It will be appreciated that many other combinations of distributed implementations of the imaging system 100 are possible without departing from the scope and spirit of the disclosure.

[0046] Moreover, in various embodiments, the various components of the imaging system 100 can be combined and / or implemented or not combined and / or implemented as desired or depending on the application or requirements. In one example, the processing component 110 can be combined with the memory component 115, the image capture component 120, the image interface 125, the display component 135, the sensing component 140, and / or the network interface 145. In another example, the processing component 110 can be combined with the image capture component 120 such that certain functions of the processing component 110 are performed by circuitry (e.g., a processor, a microprocessor, a logic device, a microcontroller, etc.) within the image capture component 120.

[0047] Figure 2 A block diagram of an example image sensor assembly 200 is shown in accordance with one or more embodiments of the present disclosure. However, not all of the depicted components can be required, and one or more embodiments can include additional components not shown in the figures. Changes can be made in the arrangement and types of components without departing from the spirit or scope of the claims as set forth herein. Additional components, different components, and / or fewer components can be provided. In one embodiment, the image sensor assembly 200 can be an FPA, such as the image capture component 120 implemented as a Figure 1

[0048] ​The image sensor assembly 200 includes a unit cell array 205, column multiplexers 210 and 215, column amplifiers 220 and 225, a row multiplexer 230, a control bias and timing circuit 235, a digital-to-analog converter (DAC) 240, and a data output buffer 245. In some aspects, operations of and / or related to the unit cell array 205 and other components can be performed in accordance with a system clock and / or a synchronization signal (e.g., a line sync (LSYNC) signal). The unit cell array 205 includes an array of unit cells. In an aspect, each unit cell can include a detector (e.g., a pixel) and an interface circuit. The interface circuit of each unit cell can provide an output signal, e.g., an output voltage or an output current, in response to a detection signal (e.g., a detection current, a detection voltage) provided by the detector of the unit cell. The output signal can be indicative of a magnitude of EM radiation received by the detector and can be referred to as image pixel data or simply image data. The column multiplexer 215, the column amplifier 220, the row multiplexer 230, and the data output buffer 245 can be used to provide the output signal from the unit cell array 205 as a data output signal on a data output line 250. The output signal on the data output line 250 can be provided to components downstream of the image sensor assembly 200, such as processing components 110 of the processing component 110, memory components 115 of the memory component 115, display components 135 of the display component 135, and / or other components, to facilitate processing, storage, and / or display of the output signal. The data output signal can be an image formed from pixel values of the image sensor assembly 200. In this regard, the column multiplexer 215, the column amplifier 220, the row multiplexer 230, and the data output buffer 245 can collectively provide a ROIC (or a portion thereof) of the image sensor assembly 200. In an aspect, the interface circuit can be considered a part of the ROIC or can be considered an interface between the detector and the ROIC. In some embodiments, components of the image sensor assembly 200 can be implemented such that the unit cell array 205 and the ROIC can be part of a single die. Figure 1 Figure 1 Figure 1

[0049] ​​​Column amplifiers 225 can generally represent any column processing circuitry suitable for a given application (analog and / or digital) and are not limited to amplifier circuitry for analog signals. In this regard, column amplifiers 225 can be more generally referred to as column processors in aspects. Signals received by column amplifiers 225, such as analog signals on an analog bus and / or digital signals on a digital bus, can be processed according to the analog or digital nature of the signals. As an example, column amplifiers 225 can include circuitry for processing digital signals. As another example, column amplifiers 225 can be a path from the array of unit cells 205 for digital signals to pass through to the column multiplexer 215 (e.g., without processing). As another example, column amplifiers 225 can include ADCs for converting analog signals to digital signals (e.g., to obtain digital count values). These digital signals can be provided to the column multiplexer 215.

[0050] Each unit cell can receive a bias signal (e.g., bias voltage, bias current) to bias the detector of the unit cell to compensate for different response characteristics of the unit cell attributable to, for example, temperature variations, manufacturing differences, and / or other factors. For example, control bias and timing circuitry 235 can generate bias signals and provide them to the unit cells. By providing each unit cell with an appropriate bias signal, the array of unit cells 205 can be effectively calibrated to provide accurate image data in response to light (e.g., visible light, IR light) incident on the detector of the unit cell. In an aspect, control bias and timing circuitry 235 can be, can include, or can be part of logic circuitry.

[0051] Control bias and timing circuitry 235 can generate control signals for addressing the array of unit cells 205 to allow access to and readout of image data from addressed portions of the array of unit cells 205. The array of unit cells 205 can be addressed to access and readout image data from the array of unit cells 205 row-by-row, although in other implementations the array of unit cells 205 can be addressed column-by-column or in other ways.

[0052] The control bias and timing circuit 235 can generate bias values and timing control voltages. In some cases, the DAC 240 can convert bias values received as or as part of data input signals on the data input signal lines 255 into bias signals (e.g., analog signals on the analog signal lines 260), which can be provided to individual unit cells through operation of the column multiplexer 210, the column amplifier 220, and the row multiplexer 230. For example, the DAC 240 can drive digital control signals (e.g., provided as bits) to appropriate analog signal levels for the unit cells. In some techniques, a digital control signal of 0 or 1 can be driven to an appropriate logic low voltage level or an appropriate logic high voltage level, respectively. In another aspect, the control bias and timing circuit 235 can generate bias signals (e.g., analog signals) and provide the bias signals to the unit cells without using the DAC 240. In this regard, some implementations do not include the DAC 240, the data input signal lines 255, and / or the analog signal lines 260. In one embodiment, the control bias and timing circuit 235 can be, can include Figure 1 The processing component 110 and / or the image capture component 120 can be Figure 1 part of or can otherwise be coupled to Figure 1 the processing component 110 and / or the image capture component 120.

[0053] In one embodiment, the image sensor assembly 200 can be implemented as part of an imaging device (e.g., the imaging device 105). In addition to the various components of the image sensor assembly 200, the imaging device can include one or more processors, memories, logic devices, displays, interfaces, optics (e.g., lenses, mirrors, beam splitters), and / or other components as can be appropriate in various implementations. In one aspect, the data output signals on the data output lines 250 can be provided to a processor (not shown) for further processing. For example, the data output signals can be an image formed from pixel values from the unit cells of the image sensor assembly 200. The processor can perform operations such as non-uniformity correction (e.g., flat field correction or other calibration techniques), spatial and / or temporal filtering, and / or other operations. The image (e.g., a processed image) can be stored in a memory (e.g., external to or local to the imaging system) and / or displayed on a display device (e.g., external to and / or integrated with the imaging system). Figure 2 The various components can be implemented on a single chip or multiple chips. Moreover, while the various components are shown as a set of separate blocks, various blocks can be combined together or Figure 2 the various blocks shown in FIG. 1 can be split into separate blocks.

[0054] Note that, inFigure 2 In particular embodiments, the array of unit cells 205 is depicted as 8x8 (e.g., 8 rows and 8 columns of unit cells. However, the array of unit cells 205 can have other array sizes. By way of non-limiting example, the array of unit cells 205 can include 512x512 (e.g., 512 rows and 512 columns of unit cells), 1024x1024, 2048x2048, 4096x4096, 8192x8192, and / or other array sizes. In some cases, the row size (e.g., the number of detectors in a row) of the array size can be different from the column size (e.g., the number of detectors in a column). Examples of frame rates can include 30 Hz, 60 Hz, and 120 Hz. In an aspect, each unit cell of the array of unit cells 205 can represent a pixel.

[0055] An infrared imaging system (e.g., a thermal camera) can be used to capture infrared image data associated with a scene using an image sensor device (e.g., an FPA). Spatial gradients in the infrared imaging system can be determined (e.g., estimated) by measuring a temporal gradient (e.g., a rate of temperature change). An infrared imaging system that heats up from the outside (e.g., when the infrared imaging system is removed from the outside of cold air to the inside of room temperature air) can exhibit a spatial thermal gradient that is related (e.g., strongly related) to the rate of temperature change. In this case, a change in ambient temperature can change the temperature of the image sensor device of the infrared imaging system relative to the temperature of another component of the infrared imaging system (e.g., a housing).

[0056] During a startup (e.g., an initial power-up) of the infrared imaging system or other internal heating processes, the infrared imaging system heats up internally. Then, for the same measured rate of change, the spatial thermal gradient can be different. As one example, the startup can end after about five minutes (e.g., a steady state begins about five minutes after the startup begins). After the startup ends, the temperatures and rates of temperature change associated with various components of the infrared imaging system are primarily caused by external factors, rather than internal heating of the infrared imaging system. Compensation can be adjusted between startup behavior and steady state behavior where temperature changes are primarily caused by external factors. In an aspect, compensation can be represented using a model (e.g., a thermal model) that characterizes temperatures associated with one or more components of the infrared imaging system. In some cases, such a model can account for thermal gradients generated by varying external temperatures. Temperature data from one or more thermal sensors can be used as input to the model.

[0057] Using various embodiments, the infrared imaging system can perform temperature compensation to provide temperature measurements with higher accuracy (e.g., higher radiometric accuracy) relative to if compensation were not performed, even during a startup mode in which the infrared imaging system exhibits significant internal heating (e.g., due to energization of various components of the infrared imaging system). The compensation can be based on temperature data from one or more temperature sensors. In some cases, each temperature sensor can be within an enclosure of the infrared imaging system. Each temperature sensor can be used to measure a temperature of a component of the infrared imaging system. A thermal gradient (e.g., a rate of change of temperature) associated with the component can be determined based on measurements of the component temperature by the temperature sensor at different points in time. In some aspects, the compensation can account for internal radiation on an image sensor device of the infrared imaging system, where such internal radiation on the image sensor device can affect temperatures determined by the infrared imaging system (e.g., temperatures of objects and / or other features / aspects of a scene). In some aspects, the compensation can allow for gradients caused by external temperature changes to be considered differently than internal changes at startup. As one example, the infrared imaging system can need to compensate for off-field radiation to provide accurate radiometric information.

[0058] Figure 3 An example system 300 for facilitating temperature compensation is shown in accordance with one or more embodiments of the present disclosure. The system 300 can be or can be part of an infrared imaging system for capturing and processing images. In one embodiment, the infrared imaging system can be the imaging system 100 of FIG. 1, can include the imaging system 100 of FIG. 1, or can be part of the imaging system 100 of FIG. 1. However, not all of the depicted components can be required, and one or more embodiments can include additional components not shown in the figures. The arrangement and type of the components can be changed, without departing from the spirit or scope of the claims set forth herein. Additional components, different components, and / or fewer components can be provided. Figure 1 Figure 1 The system 300 can be or can be part of an infrared imaging system for capturing and processing images. In one embodiment, the infrared imaging system can be the imaging system 100 of FIG. 1, can include the imaging system 100 of FIG. 1, or can be part of the imaging system 100 of FIG. 1. However, not all of the depicted components can be required, and one or more embodiments can include additional components not shown in the figures. The arrangement and type of the components can be changed, without departing from the spirit or scope of the claims set forth herein. Additional components, different components, and / or fewer components can be provided.

[0059] The system 300 includes an enclosure 305 (e.g., a camera body), one or more optical components 310, a shutter 315, an FPA 320, one or more temperature sensors 325, and a temperature compensation device 330. In one embodiment, the optical components 310, the shutter 315, the FPA 320, the temperature sensors 325, and / or the temperature compensation device 330 can be implemented using one or more processing circuits on a single chip or distributed across two or more chips.

[0060] The enclosure 305 can house the optical components 310, the shutter 315, the FPA 320, the temperature sensors 325, and / or the temperature compensation device 330. While in the example of FIG. 3, the enclosure 305 is shown as a single unitary structure, in other embodiments, the enclosure 305 can include multiple structures that are coupled together to form the enclosure 305. Figure 3 ​Optical components 310, FPA 320, temperature sensor 325, and temperature compensation device 330 are disposed within housing 305, although fewer, more, and / or different components can be disposed within housing 305. In an aspect, housing 305 can house at least optical components 310, FPA 320, and temperature sensor 325. Shutter 315 and / or temperature compensation device 330 can be disposed within housing 305 or external to housing 305.

[0061] Optical components 310 can receive electromagnetic radiation from scene 335 through aperture 340 of system 300 and pass the electromagnetic radiation to FPA 320. For example, optical components 310 can direct and / or focus electromagnetic radiation onto FPA 320. Optical components 310 can include one or more windows, lenses, mirrors, beam splitters, beam couplers, and / or other components. Optical components 310 can include components that are each formed from a material and appropriately arranged according to desired transmission characteristics (e.g., desired transmission wavelengths and / or ray transmission matrix characteristics).

[0062] Shutter 315 can be operated to selectively expose or block aperture 340. When shutter 315 is positioned to expose aperture 340, electromagnetic radiation from scene 335 can be received and directed by optical components 310. When shutter 315 is positioned to block aperture 340, electromagnetic radiation from scene 335 is blocked from optical components 310. In certain instances, shutter 315 can block aperture 340 during a calibration process, in which shutter 315 can act as a uniform black body.

[0063] FPA 320 includes a detector array and a ROIC. FPA 320 can receive electromagnetic radiation from optical component 310 and generate image data based on the electromagnetic radiation (e.g., the infrared component of the electromagnetic radiation). The image data may include infrared data values ​​(e.g., thermal infrared data values). As an example, FPA 320 may include or be coupled to ADC circuitry that generates infrared data values ​​based on infrared radiation. A 16-bit ADC circuitry can generate infrared data values ​​ranging from 0 to 65535. The infrared data values ​​can provide the temperature of different parts of a scene, such as the temperature of objects, people, and / or other aspects in scene 335. In some cases, infrared image data can be represented in an image according to a color palette, such that the visual representation value (e.g., color value or grayscale value) of each pixel in the image indicates the temperature associated with that pixel. For example, the temperature associated with an object in scene 335 can be represented in pixels (e.g., a subgroup of pixels) of the infrared image (e.g., a thermal infrared image) corresponding to the object. Infrared image data can be displayed (e.g., to a user), stored, and / or processed. In one aspect, the detector array is an infrared detector array (e.g., a microbolometer array) that detects IR radiation (e.g., thermal IR radiation). In one embodiment, the FPA 320 may be implemented by the imaging capture component 120.

[0064] During operation, when the FPA 320 receives electromagnetic radiation from scene 335, the FPA 320 may also capture radiation (e.g., internal radiation) from one or more elements / components of the infrared imaging system. Such radiation can be considered as scene-dependent / undesired radiation generated by the elements / components of the infrared imaging system and captured by the FPA 320 during its scene information capture operation. As further described herein, temperature compensation can be performed to compensate for such scene-dependent radiation, since the infrared data values ​​generated by the FPA 320 are partially based on scene-dependent radiation.

[0065] Temperature sensor 325 can measure the temperature of housing 305, optical components 310, shutter 315, FPA 320 and / or other components of system 300. Figure 3 Temperature (not shown). Each temperature sensor can be a thermistor, thermocouple, and / or other thermal sensor for measuring temperature. As an example, the infrared imaging system may be a small camera module including a single temperature sensor for measuring the temperature of the FPA 320. As another example, the infrared imaging system may include a temperature sensor for measuring the temperature of the FPA 320 and a temperature sensor for measuring the temperature of a lens (which forms at least a portion of the optical component 310). In one embodiment, the temperature sensor 325 may be implemented by or as part of the sensing component 140.

[0066] The temperature compensation device 330 can receive image data (e.g., infrared data values) generated by the FPA 320 and temperature data from the temperature sensor 325. The temperature compensation device 330 can perform temperature compensation on the image data to account for (e.g., mitigate) any radiation (e.g., off-field radiation, internal radiation, etc.) that the FPA 320 can receive that can reduce the accuracy of the radiation measurements of the image data generated by the FPA 320. In one embodiment, the compensation can be adjusted between compensation when the infrared imaging system exhibits start-up behavior (e.g., is in a start-up mode) and compensation when the infrared imaging system exhibits steady-state behavior (e.g., is in a steady-state mode).

[0067] In an aspect, the temperature compensation device 330 can provide temperature measurements / values of one or more objects / people / features in the scene as output. For example, the temperature measurements / values can be provided using numerical values. In some cases, alternatively or additionally, the temperature compensation device 330 can represent such temperature measurements / values in the image, where the temperature values associated with each pixel of the image are represented using color values or grayscale values according to a color palette. The image generated by the temperature compensation device 330 by performing temperature compensation on the image data from the FPA 320 can be referred to as a temperature compensated image. More generally, the output generated by the temperature compensation device 330 by performing temperature compensation on the image data from the FPA 320 can be referred to as temperature compensated image data or temperature compensated output. The output of the temperature compensation device 330 can be provided for further processing (e.g., noise reduction processing, fusion with images of the same or other wavebands, etc.), storage, and / or display.

[0068] In one embodiment, the FPA 320 and the temperature compensation device 330 can be collectively implemented by the imaging capture component 120, where the output of the temperature compensation device 330 is referred to as temperature compensated image data. In some cases, the temperature compensated image data can be indicative of temperatures of objects in the scene 335 determined using temperature compensation in part. In some cases, the temperature compensated image data can be represented in a temperature compensated image. In another embodiment, the FPA 320 can be implemented by the imaging capture component 120 and the temperature compensation device 330 can be implemented by the processing component 110. In such an embodiment, the image data from the imaging capture component 120 can be provided to the processing component 110 (e.g., via the image interface 125) and temperature compensation performed on the image data by the processing component 110 to obtain temperature compensated image data. In some cases, a user of the infrared imaging system can have access to the image data generated by the FPA 320 (e.g., prior to compensation) and the temperature compensated image data. In other cases, a user of the infrared imaging system can only have access to the temperature compensated image data.

[0069] In one or more embodiments, the temperature compensation performed by the temperature compensation device 330 can be based on the temperature of one or more components of the infrared imaging system whose radiation is on the FPA 320. As the components radiate onto the FPA 320, the temperature of the components can affect the image data captured by the FPA 320. In some aspects, such components are located within the housing 305 of the infrared imaging system. Thus, the temperature of such components provides an internal temperature of the infrared imaging system.

[0070] For some of these components, the temperature of the components can be directly measured using one or more temperature sensors 325. As a non-limiting example, a temperature sensor can be disposed on the FPA 320, the housing 305, and / or a lens that directs light to the FPA 320 (e.g., a portion of the optical components 310). Other components can be without a temperature sensor to measure their temperature. The temperature of one or more of these other components can be determined (e.g., estimated, modeled) based in part on temperature data (e.g., temperature measurements) from the temperature sensors 325. As a non-limiting example, these components can include the housing 305 (e.g., when the housing 305 is without a temperature sensor to measure its temperature), a lens that directs light to the FPA 320, and / or the shutter 315. In certain instances, a temperature sensor can not be disposed on every component itself due to space considerations (e.g., limited space around the components and / or within the housing, obstructing the optical path to the FPA 320 by disposing a temperature sensor on the component, etc.), power considerations (e.g., each disposed temperature sensor requires power to operate), and / or other considerations. Thus, using various embodiments, the temperature compensation performed by the temperature compensation device 330 can be based on temperature data (e.g., temperature measurements) from one or more components of the temperature sensors 325 and the temperature of one or more other components determined (e.g., through appropriate modeling) based on the temperature data from the temperature sensors 325.

[0071] As one non-limiting example, a general model to describe the spatially varying internal temperature of the infrared imaging system as a function of the temperature data from the temperature sensors 325 and the temporal variation of the temperature data (e.g., rate of change of the temperature measurements) can be provided by the following equation:

[0072]

[0073] where T i is the temperature of the i-th component of the infrared imaging system, N > 1 is the number of temperature sensors, A ij and B i are used to determine T i from the temperature measurements TS jThe model parameters of the function, where j = 0, ..., N-1, and C ij It is to determine the temperature T of the i-th component of the infrared imaging system. i The rate of change dTS of temperature measurements provided by each of N temperature sensors j The model parameters related to / dt. At this point, the temperature T of the i-th component of the infrared imaging system. i The temperature of the infrared imaging system components, measured by temperature sensor 325, depends on the temperature of the components and its rate of temperature change. Other environmental conditions besides ambient temperature, such as humidity, may affect radiation. In some respects, the model may take into account humidity levels and / or other atmospheric / environmental measurements that may affect internal radiation.

[0074] As an example, an infrared imaging system could be a small camera module comprising a single temperature sensor for measuring the temperature of the FPA 320. The temperatures of one or more other components (e.g., housing 305, shutter 315, and / or lens) could be determined at least based on the temperature measurement of the FPA 320 by the single temperature sensor. In this regard, referring to equation (1), the temperature measurement TS j From a single temperature sensor, and the rate of change dTS j / dt can be derived from a temperature measurement, and its temperature T i The components that are identified / modeled may include housing 305, shutter 315, and / or lens.

[0075] As another example, the infrared imaging system may include a temperature sensor for measuring the temperature of the FPA 320 and a temperature sensor for measuring the temperature of the lens. The temperatures of other components (e.g., housing 305 and / or shutter 315) can be determined at least based on temperature measurements of the FPA 320 and the lens. In this example, referring to equation (1), the temperature measurement TS j From two temperature sensors and the rate of change dTS j / dt is derived from a temperature measurement, and its temperature T i The identified / modeled components may include housing 305 and / or shutter 315. In some cases, such as when housing 305 and shutter 315 are close to each other and / or implemented using the same or similar materials, the temperature T of housing 305 and shutter 315... i They can be assumed to be the same. Note that in some cases, the temperature of shutter 315 (e.g., whether measured using a temperature sensor or modeled) can be used as part of the calibration process.

[0076] On the one hand, parameter A ij and B i It is a constant parameter (e.g., it remains constant over time) while parameter C ijmay be a time-dependent parameter. In this regard, the parameter C ij may vary continuously from a start-up behavior to a steady-state behavior. As an example, in addition to start-up, the internal heating process can be caused by a change in operating mode of the infrared imaging system, such as a change in frame rate associated with the FPA 320, which can result in a change in internal heating. As another example, the internal heating process can be caused by turning on / off or otherwise changing a power dissipating electronic device (e.g., an image processor proximate to the FPA 320). As a non-limiting example, the parameter C ij may be provided by:

[0077]

[0078] where t = 0 can be the start time of the start-up or other internal heating process. In some cases, the parameter a can be determined empirically to allow C init to transition appropriately between C final As an example, for a smooth transition between C init and C final , the parameter a is selected to be 2. The parameter t0provides a characteristic time constant associated with the transition from start-up to steady-state. C final and C init The ratio between C final and C init may characterize the difference between start-up heating and heating / cooling due to external causes (e.g., changes in ambient temperature). As one example, depending on the infrared imaging system being modeled, the ratio between C final and C init may be 5 or greater, thus showing the importance of distinguishing between start-up heating and external heating / cooling for accurate compensation.

[0079] As another non-limiting example, the parameter C ij may be provided by:

[0080]

[0081] where t = 0 can be the start time of the start-up or other internal heating process. In this regard, the parameter C ij is a piecewise function, where the parameter t0provides a characteristic time constant associated with the transition from start-up to steady-state.

[0082] As another non-limiting example, the parameter C ij may be provided by:

[0083]

[0084] Where t = 0 can be the start time of startup or other internal heating processes, parameter t0 can provide a characteristic time constant associated with the transition from startup to steady state, and parameter τ can determine the time scale of the transition. In equation (4), for lower values ​​of t, C ij (t)≈C init And for higher values ​​of t, C ij (t)≈C final .

[0085] As mentioned above, many different models can be used to describe parameter C. ij The temporal correlations provided in equations (1) through (4) allow for adaptive models (e.g., equations (1), (2), and (4) are continuously adaptive models) that differentiate and transition between startup and steady-state behavior. The amount of time an infrared imaging system spends operating in startup and steady-state modes can be partly based on the architecture / components that make up the infrared imaging system. In some cases, this amount of time can be further based on the ambient temperature at which the infrared imaging system is turned on. As an example, an infrared imaging system may primarily exhibit startup behavior of approximately four to six minutes after being turned on.

[0086] Parameter A can be determined during the manufacturing and / or calibration process of the infrared imaging system or a part thereof. ij B i and C ij (including C, which depends on the model) init C final , t0 and τ). In some cases, parameter A ij B i and / or C ij This can be a manufacturing parameter associated with a single camera or a row of cameras. As an example, the calibration of an infrared imaging system can be based on capturing images of an object with a known temperature using the infrared imaging system and / or components of the infrared imaging system being maintained at a known temperature. During steady-state operation (e.g., after startup or other internal heating processes), parameter C... ij Can be combined with constant parameter A ij and B i Together they are considered constant parameters (e.g., for equation (2), for large t, the term exp(-t / t0)). a =0, therefore for large t, C ij (t)≈C final Therefore, the model provided by equation (1) is based on the constant parameter A. ij B i and C ij In certain circumstances, it can be accurate.

[0087] In a non-restricted example, parameter Aij B i and C final This can be determined during steady-state operation. To achieve steady-state operation, the infrared imaging system can be switched on and kept on, causing the infrared imaging system to exhibit steady-state behavior where temperature changes are primarily caused by external factors. At this time, C... ij (t)≈C final External temperatures, such as ambient temperature and / or the temperature of objects in the scene, can be known and / or controllable to facilitate parameter A. ij B i and C final The determination of parameter A. ij B i and C final Afterwards, the infrared imaging system can be connected and its startup process examined to determine C. init Parameter A can be determined using [the specified parameter]. ij B i and / or C ij Other methods. For example, in some cases, parameter A can be determined at least partially during startup. ij and / or B i .

[0088] Although in the foregoing, parameter A ij and B i As time is constant and parameter C ij It is time-related, but in some cases, parameter A ij and / or B i It can be time-related. Furthermore, although the model provided in equation (1) offers three terms, each related to parameter A... ij B i Or C ij One of the initial conditions may be associated with an infrared imaging system, but other models may involve fewer or more than three terms and / or involve one or more terms associated with multiple parameters. In some cases, parameters may have different values ​​depending on the initial conditions associated with the infrared imaging system. As an example, if the infrared imaging system is turned on at a colder ambient temperature, it may operate for a longer time in startup mode compared to when it is turned on at a warmer ambient temperature. Different initial conditions can be used to calibrate the infrared imaging system, and parameters such as parameter t0 can have different values ​​depending on the initial conditions. For example, during startup, the infrared imaging system may determine the initial conditions and select parameter t0 (determined during calibration) as the closest initial condition to the initial conditions determined by the infrared imaging system. In other cases, models can be defined such that the parameters are independent of the initial conditions.

[0089] Performing temperature compensation by considering modeling of start-up behavior and steady-state behavior, such as by using the models provided by Equations (1) and (2), allows the infrared imaging system to provide high radiometric accuracy (e.g., temperature determinations) even during start-up. In this regard, a user of the infrared imaging system can avoid having to wait until the start-up behavior has ended (which can take five minutes or more depending on the infrared imaging system) to obtain accurate radiometric measurements from the infrared imaging system.

[0090] Figure 4 A flowchart illustrating an example process 400 for facilitating temperature compensation in accordance with one or more embodiments of the present disclosure is shown. Although the process 400 is described herein primarily with reference to the system 300 of Figure 3 , the process 400 can be performed with respect to other systems to facilitate temperature compensation. Note that one or more operations in Figure 4 may be combined, omitted, and / or performed in a different order as desired.

[0091] At block 405, the FPA 320 captures radiation (e.g., infrared radiation). The radiation can include radiation from the scene 335 as well as radiation from one or more elements of the infrared imaging system. The radiation from the elements can be considered non-scene related radiation / undesired radiation that is captured by the FPA 320 during the operation of the FPA 320 capturing scene information. Temperature compensation can be performed to compensate for such non-scene related radiation because the infrared data values generated by the FPA 320 are based in part on the non-scene related radiation. As a non-limiting example, the elements can include the housing 305, the shutter 315, or a lens element (e.g., a portion of the optical components 310). In certain instances, in order to reach the FPA 320, the radiation (e.g., from the scene 335) can propagate through an optical path of the infrared imaging system formed by one or more optical components 310.

[0092] At block 410, the FPA 320 generates image data (e.g., infrared image data) in response to the radiation. In some cases, the FPA 320 and / or circuitry coupled to the FPA 320 can convert the radiation into an electrical signal (e.g., a voltage, a current, etc.) and generate the image data based on the electrical signal. The image data can include pixel values. The pixel values can be represented in terms of digital count values generated from the electrical signal obtained from the converted radiation. For example, in cases where the FPA 320 includes or is otherwise coupled to ADC circuitry, the ADC circuitry can generate the digital count values based on the electrical signal. For ADC circuitry that can represent the electrical signal using 14 bits, the digital count values can range from 0 to 16383. In some cases, the FPA 320 can represent the image data in the image in terms of a palette. A given pixel of the image can have a visual representation value (e.g., a color value or a grayscale value) that is indicative of a temperature of the pixel. For example, a temperature associated with an object in the scene 335 can be represented in a pixel of an infrared image (e.g., a thermal infrared image) formed from the image data corresponding to the object. This temperature associated with the object can be referred to as an uncompensated temperature.

[0093] At block 415, the temperature sensor 325 determines a temperature of the FPA 320. In some cases, the temperature sensor 325 can determine a temperature of other components of the infrared imaging system (e.g., the housing 305 and / or the shutter 315). As one example, the temperature sensor 325 can include only a temperature sensor disposed on the FPA 320 and used to measure a temperature of the FPA 320. As another example, the temperature sensor 325 can include at least a temperature sensor disposed on the FPA 320 and used to measure a temperature of the FPA 320 and a temperature sensor disposed on the shutter 315 and used to measure a temperature of the shutter 315.

[0094] At block 420, the temperature compensation device 330 determines a temperature associated with an element of the infrared imaging system based on the temperature data from the temperature sensor 325. In some cases, for a given element, the temperature associated with the element can be based on a temperature of the FPA 320 and a rate of change of the temperature of the FPA 320. In some cases, the temperature data can include temperatures and rates of change of temperatures of other components of the infrared imaging system (e.g., measured by the temperature sensor) other than the FPA 320.

[0095] In one embodiment, the compensation can be adjusted between compensation when the infrared imaging system exhibits start-up behavior (e.g., is in a start-up mode) and compensation when the infrared imaging system exhibits steady-state behavior (e.g., is in a steady-state mode). In one aspect, to adjust between compensation in the start-up mode and compensation in the steady-state mode, the temperature associated with the element can be modeled according to equation (1). Note that whether the temperature of an element of the infrared imaging system is modeled for the purposes of temperature compensation can be based on the relative contribution of the radiation of the element on the FPA 320. If an element is determined (e.g., by simulation, modeling, or otherwise) to have a negligible impact on the radiation captured by the FPA 320, the element can be ignored for the purposes of temperature compensation. Whether an element is considered to have a negligible impact is generally dependent on the application (e.g., radiation measurement accuracy requirements, power requirements), computational resources, etc.

[0096] At block 425, the temperature compensation device 330 determines a temperature associated with an object in the scene 335 based on the image data, the temperature associated with the element, and the temperature of the FPA 320. This temperature associated with the object can be referred to as a compensated temperature. In this regard, the temperature associated with the object can be referred to as a temperature compensation output of the temperature compensation device 330. The temperature compensation performed by the temperature compensation device 330 can be to compensate for radiation from one or more elements of the infrared imaging system captured at block 405. In certain instances, the radiation from the scene 335 can be considered to be useful or target image data, while the radiation from the one or more elements can be considered to be noise or undesirable image data. The temperature associated with the object can be further based on a rate of change of the temperature of the FPA 320. In certain instances, the temperature associated with the object can be based on temperatures and rates of change of temperatures of other components of the infrared imaging system. In one aspect, the temperature compensation device 330 can generate temperature compensated image data. In certain instances, the temperature compensated image data can be represented in an image, which can be referred to as a temperature compensated image.

[0097] While the foregoing describes compensating infrared image data in real-time or near real-time with the capture of the infrared image data, the infrared image data can be stored (e.g., in the memory component 115) for retrieval and compensation at a later time. In certain instances, the infrared image data can be associated with a timestamp indicating a time at which the infrared image data was captured and stored. Temperature data captured at or around the time indicated by the timestamp can also be stored. With the infrared image data and corresponding temperature data, the infrared image data can be compensated at the later time. In certain instances, compensation performed at or around the time of capture of the infrared image data can be replaced or in addition to, compensation performed at the later time. As an example, a thermal model provided by, for example, equations (1) and (2) can be refined (e.g., to determine A ij , B iand / or C ij ), so that the compensation at later times can be associated with a higher accuracy of the radiation measurements than the compensation using the thermal model before the improvement.

[0098] Where applicable, various embodiments provided by the present disclosure can be implemented using hardware, software, or combinations of hardware and software. Also where applicable, the various hardware components and / or software components set forth herein can be combined into composite components comprising software, hardware, and / or both, without deviating from the spirit of the present disclosure. Where applicable, the various hardware components and / or software components set forth herein can be divided into sub-components of software, hardware, and / or both, without deviating from the spirit of the present disclosure. Also where applicable, it is contemplated that software components can be implemented as hardware components, and vice versa, without deviating from the spirit of the present disclosure.

[0099] Software, according to the present disclosure, such as non-transitory instructions, program code, and / or data, can be stored on one or more non-transitory machine-readable media. It is also contemplated that software identified herein can be implemented using one or more general purpose or special purpose computing systems, with or without coupled memory, such as, for example, computer systems based on microprocessors, microcontrollers, programmable consumer electronics, minicomputers, mainframe computers, and the like. Also, software can be implemented in a computing system that includes one or more computers operating in a networked environment. Where applicable, the order of execution or the order of performance of the various steps described herein can be altered, the various steps combined into composite steps, and / or the various steps split into sub-steps, to provide a software implementation that is within the scope of the present disclosure.

[0100] The foregoing description is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. The embodiments described above are described as examples only, and are not intended to limit the scope of the disclosure to these and other forms described or suggested herein. Various alternatives and / or modifications of the embodiments described above can be contemplated as being within the scope of the present disclosure as described and defined by the appended claims. The scope of the disclosure is thus indicated by the appended claims, rather than the foregoing description.

Claims

1. An infrared imaging system, comprising: A focal plane array (FPA) configured to capture radiation from a scene and generate infrared image data based on the radiation, wherein, during the capture of radiation from the scene, the focal plane array also captures radiation from elements associated with the infrared imaging system; A first temperature sensor, configured to determine the temperature of the focal plane array; and Processing circuit, the processing circuit being configured to: The temperature associated with the element is determined at least in part based on the product of the temperature of the focal plane array and a first parameter, and the product of the rate of temperature change of the focal plane array and a second parameter; and Based on the infrared image data, the temperature associated with the element, and the temperature of the focal plane array, the temperature associated with the object in the scene is determined.

2. The infrared imaging system according to claim 1, wherein, The component includes a housing, wherein the focal plane array and the first temperature sensor are disposed within the housing, and wherein the processing circuitry is configured to determine the temperature associated with the component based on whether the infrared imaging system is operating under startup conditions or under steady-state conditions.

3. The infrared imaging system according to claim 1, wherein, The element includes a housing, shutter, or lens, wherein the infrared image data indicates a temperature estimate associated with the object, and wherein the processing circuitry is configured to determine the temperature associated with the object by compensating for radiation from the element, at least in part, based on the temperature of the focal plane array, to obtain the temperature of the object.

4. The infrared imaging system according to claim 1, wherein, At least the second parameter is a time-dependent parameter.

5. The infrared imaging system according to claim 1, further comprising a housing, wherein: The element includes a shutter; The processing circuitry is also configured to determine the temperature associated with the housing, in part based on the temperature of the focal plane array; and The processing circuitry is configured to also determine the temperature associated with the object based on the temperature associated with the housing.

6. The infrared imaging system of claim 1, further comprising a housing and a second temperature sensor configured to determine the temperature of the housing, wherein, The element includes a lens, and the processing circuitry is configured to determine a temperature associated with the lens based on the temperature of the housing and the temperature of the focal plane array.

7. The infrared imaging system according to claim 6, wherein, The processing circuit is configured to determine the temperature associated with the lens based on the temperature of the housing, the rate of temperature change of the housing, the temperature of the focal plane array, and the rate of temperature change of the focal plane array.

8. The infrared imaging system according to claim 7, wherein, The temperature associated with the lens is based on the product of the temperature of the focal plane array and the first parameter, the product of the temperature change rate of the focal plane array and the second parameter, the product of the temperature of the housing and the third parameter, and the product of the temperature change rate of the housing and the fourth parameter.

9. The infrared imaging system according to claim 1, wherein, The focal plane array includes a microbolometer array configured to capture radiation from the scene, and wherein the focal plane array also captures radiation from the element during the capture of radiation from the scene.

10. A method for temperature compensation of infrared image data, the method comprising: Radiation from elements and from the scene is captured using a focal plane array (FPA); Infrared image data is generated based on the radiation using the focal plane array; Determine the temperature of the focal plane array; The temperature associated with the element is determined at least in part based on the product of the temperature of the focal plane array and a first parameter, and the product of the rate of change of the temperature of the focal plane array and a second parameter. and Based on the infrared image data, the temperature associated with the element, and the temperature of the focal plane array, the temperature associated with the object in the scene is determined.

11. The method according to claim 10, wherein, The element includes a housing of an infrared imaging system, wherein the focal plane array is disposed within the housing, and wherein the temperature associated with the element is based on whether the infrared imaging system is operating under startup conditions or under steady-state conditions.

12. The method according to claim 10, wherein, At least the second parameter is a time-dependent parameter.

13. The method of claim 10, further comprising determining a temperature associated with the housing, wherein, The element and the focal plane array are disposed within the housing, and the temperature associated with the object is also based on the temperature associated with the housing.

14. The method of claim 10, further comprising measuring the temperature of the housing using a temperature sensor, wherein, The element, the focal plane array, and the temperature sensor are disposed within the housing, and the temperature associated with the element is based on the temperature of the housing and the temperature of the focal plane array.

15. The method according to claim 14, wherein, The temperature associated with the element is based on the temperature of the housing, the rate of temperature change of the housing, the temperature of the focal plane array, and the rate of temperature change of the focal plane array.

16. The method according to claim 15, wherein, The temperature associated with the element is based on the product of the temperature of the focal plane array and the first parameter, the product of the temperature change rate of the focal plane array and the second parameter, the product of the temperature of the housing and the third parameter, and the product of the temperature change rate of the housing and the fourth parameter.

Images