Method for full-resolution estimation of temperature emissivity by multispectral thermal infrared imaging

Abstract

The invention relates to a method for processing TIR images, comprising the steps of: generating training data (MDD) from a physical model of a multi-spectral-channel bolometric sensor, the physical model providing a radiance measurement as a function of temperature and emissivity values; training a regression module (TRD) estimating temperature (TMP) and emissivity (EMSi) images for each spectral channel (i), from the training data; training a rendering module (MRIR) configured to produce a full-resolution multispectral radiance image; training a processing chain comprising the rendering and regression modules, and a refinement module (PRTR, PSTR); acquiring a radiance image (LM[1-C]) for each spectral channel; and processing the image acquired by the trained processing chain to obtain a temperature image and an emissivity image per spectral channel. Figure 1

Description

Title of the invention: Method for full-resolution estimation of temperature emissivity by multispectral thermal infrared imaging. Technical field

[0001] The present invention relates to infrared thermal imaging (IRT). This is a process that generates an image of a scene by measuring the infrared radiation emitted by that scene. Since the amount of thermal radiation depends on the temperature, it is possible to see temperature variations in the captured scene. Prior art

[0002] IR radiation can be measured in different ways, depending on the type of sensor. For example, a bolometer measures radiant heat using a material whose electrical resistance depends on temperature. More precisely, the incident IR flux causes a change in electrical resistance from which radiation parameters can be deduced.

[0003] Physically, radiation is determined by variables that are initially unknown, namely a temperature variable and emissivity variables of the observed object, representing the efficiency with which an object emits thermal radiation. To study IR radiation and exploit this radiation in an application system, it can be useful to precisely determine the temperature and emissivity from the measured radiation. However, the problem of temperature and emissivity separation (TES) is complex, particularly due to the non-linear relationship between temperature and radiance. For the sake of simplification, most existing work assumes that the emissivity is essentially constant in each spectral channel i considered.

[0004] Under this assumption, for a given temperature T, the radiance Lmes i(T ) measured in the spectral channel i can be calculated using the following simplified model:

[0005] Lmes, i ( T ) =EiLCN j( T ) + ( 1 -E; ) L( x / PJ ( 1 )

[0006] in which Σi represents the average emissivity of the object in channel i and Te represents the temperature of the environment. The radiance emitted directly by the target Lcnj(T) and the radiance of the environment reflected by the target LCNi(Te) can be calculated using Planck's equation:

[0007] (2)

[0008] with h = 6.626x1034 m2.kg / s, c = 3xl08 m / s, k = 1.38x1023 m2.kg / s2 K 1 and f(X) is the spectral response of a filter covering the band of wavelengths extending from Xmin to Xmax defining the spectral channel i.

[0009] It turns out that multispectral sensors with M spectral channels Xi=i...M can only provide M TES measurements for M+l variables (M emissivities and a temperature T). The problem thus appears ill-posed because the number of equations linking the variables to be determined is less than the number of these variables. Indeed, there is no one-to-one correspondence between T,ei=i...M and Lmes>i (T). There is therefore an infinite number of predecessors of Xi=i,M. To circumvent this lack of equations, most existing TES algorithms inherently add constraints. Due to their iterative and non-deterministic nature, they are therefore slow and can hardly be deployed in end-of-chain devices that can integrate sensors. Furthermore, these algorithms process each pixel of the scene separately, that is, without taking into account information from the neighboring region.Therefore, they cannot guarantee to produce images free of artifacts or aliasing effects of a visual quality pleasing to the human eye (i.e., with sharp edges and uniformity within the same object).

[0010] Temperature-Emissivity Separation (TES) is a crucial step when using TIR sensors to separate the object's surface temperature from its emissivity. As mentioned above, in the context of a multispectral TIR image with M spectral measurement channels, one always ends up with an underdetermined system of equations with only M equations for M + 1 unknowns (one temperature equation and M emissivity equations over M spectral channels, based on the assumption of constant emissivity per band). Other assumptions could be made here regarding the variation of emissivity as a function of wavelength (other than piecewise or bandwise constant). For example, this variation could be modeled by the linear combination of reference curves (which can therefore be analyzed by a transform).These reference "curves" could be defined either manually or by machine learning based on a database of typical emissivity of the materials considered.

[0011] Different approaches have been proposed to solve this problem, which can be divided into two groups: iterative algorithms without data and algorithms based on data-guided learning.

[0012] In dataless iterative algorithms, an additional constraint (or prior assumption) is introduced to eliminate underdetermination. This additional constraint generally applies to the emissivity, that is, to the range of possible values ​​of the emissivity given the range of wavelengths. Some of these algorithms apply the normalized emissivity method (NEM). In particular, [1] uses an empirical relationship between a range of emissivities and a minimum value for a multichannel observation. Starting from an assumed emissivity value, these algorithms estimate a relative emissivity from the measured radiation and then calculate a minimum emissivity value using an empirical relationship established from laboratory experiments. This yields an emissivity value for each spectral channel and a temperature value. This process is then repeated iteratively with a new assumed emissivity value until the obtained values ​​converge.

[0013] Subsequently, new iterative algorithms were developed within the framework of maximum entropy [2]. This involves calculating the probability density distribution (PDF: "Probability Density Distribution") of the random variables, namely temperature and emissivities, in order to derive the estimated values ​​of these variables.

[0014] Data-driven learning algorithms primarily rely on the use of neural networks and a specific training dataset to estimate temperature and emissivity from radiance measurements. These algorithms mainly focus on the case of extracting Earth surface temperature using hyperspectral measurements. Aires et al. [3] propose a single-hidden-layer multilayer perceptron (MLP) to extract surface or atmospheric temperature from a large number of radiance measurements. Chen et al. [4] use principal component analysis (PCA) to compress and remove noise from hyperspectral TIR data before feeding it into a single-hidden-layer MLP.More recently, Ye et al. [5] exploit a deep neural network structure comprising a bidirectional recurrent network with long short-term memory (LSTM) combined with several 1D convolutional layers and fully connected layers, in order to exploit correlations between hyperspectral TIR channels. Finally, [6] presents a heat-assisted detection and ranging (HADAR) system, which allows for the estimation of temperature, emissivity, and a texture image from a hyperspectral thermal radiation cube.

[0015] Recently, TIR images have received increased attention as an alternative to RGB images to improve the performance of perception algorithms in adverse weather and lighting conditions. Several studies have focused on rendering tasks such as denoising or super-resolution to improve the visual quality of TIR images. In particular, Saragadam et al. [7] propose an image processing chain for cameras thermal imaging combines a physical model of an uncooled bolometric camera with a deep convolutional neural network to address denoising and super-resolution issues. This processing chain relies on capturing multiple images of the scene and estimating the scene's radiance flux using a deep regularized neural network. Shyam et al. [8] propose a lightweight super-resolution network to increase the thermal image resolution, which is then fused with an RGB image to improve the performance of the object detection algorithm in adverse weather conditions. In this invention, the radiance image rendering module can apply various operations (e.g., denoising, demosaicing, or super-resolution) to the input multispectral TIR images to produce full-frame, correct radiance images ready for the TES operation.[9] proposes a multi-level feature attention network that is antagonistically trained to denoise infrared images. It is worth mentioning that existing work mainly proposes thermal image rendering networks, either autonomously or as a component of a multimodal perception algorithm, to process a single input thermal image.

[0016] There are also blind image quality assessment (BIQA) algorithms designed to automatically evaluate the perceived quality of a single image without any reference, that is, to assess the image quality without comparing it to a known, high-quality reference image. By determining the quality of an image, it is possible to make informed decisions about its use or suitability for specific tasks, or to adjust a processing chain to improve the quality of the output image. Most BIQA algorithms are initially proposed for RGB images. However, some work focuses on evaluating the quality of thermal images. Agaian et al.

[10] introduce new measures for evaluating the quality of thermal images based on the human visual system, the information system, and distribution.Teutsch et al

[11] propose quality measures covering exposure, contrast, noise visibility, and temporal inconsistency attributes for tone mapping of thermal infrared videos. Erlenbusch et al

[12] use an existing BIQA algorithm in a fog-aware density estimator (FADE: "Fog Aware Density Evaluator") to denoise TIR images.

[0017] It is therefore desirable to improve existing multispectral image processing algorithms by proposing a method capable of rapidly providing high-quality images (pleasing to human observers) without requiring significant computing power. It is also desirable not to limit oneself to a specific context such as the Earth's surface or the atmosphere, as some do. processes described previously. In addition, it is also desirable to directly produce absolute thermographic images. Résumé

[0018] Embodiments relate to a method for processing infrared thermal images, IR, comprising steps of: generating training data from a physical model of a bolometric sensor comprising several spectral channels, the physical model providing for each spectral channel a radiance measurement as a function of temperature and emissivity values ​​in the spectral channel or as a function of coefficients of a linear combination of several spectral response curves; training a regression module comprising a forward-predictive neural network to estimate a temperature image and an emissivity image for each spectral channel or coefficient, from the training data; training an image rendering module configured to produce a full-resolution multispectral radiance image having appropriate visual quality in a luminance range, from a database of real images;to train a processing chain comprising the rendering module, the regression module, and at least one refinement module disposed between the rendering module and the regression module and / or at the output of the regression module, the rendering module and the regression module being fixed or adjusted by regularization during training; to acquire by a processor, a radiance image provided by a multispectral bolometric sensor, the radiance image comprising for each pixel a radiance value for each spectral channel or each coefficient; and to process the image acquired by the trained processing chain, implemented by the processor, to obtain a temperature image and an emissivity image per spectral channel or per coefficient.

[0019] According to one embodiment, the regression module includes a temperature image estimation neural network which is trained and then used to train an emissivity image estimation neural network for each spectral channel or for each coefficient.

[0020] According to one embodiment, the processing chain comprises a first refinement module disposed between the rendering module and the regression module and a second refinement module disposed at the output of the regression module, the first and second refinement modules being trained at the same time during the training of the processing chain.

[0021] According to one embodiment, the radiance image rendering module is configured to separately process each radiance image for a spectral channel provided by the sensor and produce a full-resolution radiance image for the spectral channel with appropriate visual quality in the luminance domain.

[0022] According to one embodiment, the refinement module is configured to process each pixel of the radiance images by taking into account the neighboring pixels, to remove any artifacts, and improve the visual quality of the temperature / emissivity images.

[0023] According to one embodiment, the image rendering module performs at least one of the following processing: a demosaicing process, a denoising process, a resolution upscaling process, and a realignment process between spectral channels.

[0024] According to one embodiment, the training of the processing chain is carried out in a self-supervised manner or without using reference temperature and emissivity images.

[0025] According to one embodiment, the training of the processing chain is carried out using a fidelity or loss of similarity function to constrain the behavior of the refinement module not to remove global structures in the images compared to unrefined images provided by the regression module.

[0026] According to one embodiment, the refinement module is configured to perform smoothing and / or denoising in uniform areas and edge sharpening of temperature and emissivity images provided by the regression module, by forcing the output images of the processing chain to get closer to the output images of the regression module.

[0027] According to one embodiment, the processing chain comprises: a panchromatic estimation process producing a panchromatic radiance image from an input or output image of the regression module, and / or a false-color thermal image estimation process receiving the input and / or output image of the regression module, and / or output image of the refinement module.

[0028] According to one embodiment, the method includes a step of merging the temperature image and / or emissivity images with images from an image sensor in the visible domain.

[0029] According to one embodiment, the processing chain includes a perception module receiving images from the refinement module, and which provides an inference decision.

[0030] Embodiments may also relate to a device for generating multispectral temperature and emissivity images, comprising: a multispectral radiance sensor, and a processor configured to implement the steps of the process as defined above.

[0031] According to one embodiment, the sensor is of the infrared thermal bolometer (IRT) type, comprising a filter disposed in a focal plane of the sensor and having a mosaic pattern of 2 x 2 filter elements, each filter element of the pattern corresponding respectively to one of the spectral channels.

[0032] Embodiments may also relate to a computer-readable recording medium comprising instructions which, when executed by a computer, lead the computer to implement the process as defined above. Brief description of the figures

[0033] The present invention will be better understood with the aid of the following description of exemplary embodiments with reference to the accompanying figures, in which identical reference numerals correspond to structurally and / or functionally identical or similar elements, and in which:

[0034] [Fig-1] Figure 1 schematically represents an acquisition and Multispectral TIR image processing, according to one embodiment,

[0035] [Fig.2] Figure 2 schematically represents a module of the device acquisition, producing a temperature estimate from multispectral TIR image data, according to one embodiment,

[0036] [Fig. 3] Figure 3 schematically represents a module producing an emissivity estimate from multispectral TIR image data, according to one embodiment,

[0037] [Fig. 4] Figure 4 schematically represents a module of the device acquisition, performing demosaicing of multispectral TIR images, according to one embodiment,

[0038] [Fig. 5] Figure 5 schematically represents an image rendering module of the acquisition device, according to one embodiment,

[0039] [Fig. 6] Figure 6 schematically represents part of the image editing module, according to one embodiment,

[0040] [Fig.7] Figure 7 schematically represents part of the retouching module of image, according to a method of realization,

[0041] [Fig.8] Figure 8 schematically represents a refinement module of the acquisition device, according to one embodiment,

[0042] [Fig.9] Figure 9 schematically represents an image processing chain Multispectral IR shooting, according to another embodiment,

[0043] [Fig. 10] Figure 10 schematically represents a multispectral TIR image processing chain, according to another embodiment,

[0044] [Fig. 11] Figure 11 schematically represents a multispectral TIR image processing chain, according to another embodiment,

[0045] [Fig. 12] Figure 12 schematically represents, in perspective, a multispectral radiance image sensor, according to one embodiment,

[0046] [Fig. 13] Figure 13 schematically represents the device for acquiring and processing multispectral TIR images, according to one embodiment. Detailed description

[0047] Figure 1 shows a device for acquiring and processing multispectral radiance images, according to one embodiment. The device comprises a PRC processor and an SNS sensor for multispectral radiance LM[1-C] images with C spectral channels, connected to the processor. The PRC processor implements a processing method based on deep neural networks generated by self-supervised learning dedicated to rendering full-resolution TMP temperature and EMS emissivity images produced from the LM[1-C] images. The processing method is configured to estimate both a TMP temperature image and an EMSi emissivity image per spectral channel i, assuming constant emissivity in each spectral channel considered.The processing method relies on the central concept of deep learning, using different neural networks and learning strategies (supervised and self-supervised) for absolute thermography applications. It also relies on the concept of image quality evaluation to produce images pleasing to human observers.

[0048] The PRC processor includes a processing chain comprising four modules, namely: a low-level radiance image rendering MRIR module, a regression neural network TRG module, and two refinement modules, PRTR and PSTR, which can be placed before or after the TRG regression model. In practice, a processing chain may contain only one refinement module, and the role of the MRIR module is configurable, depending on the nature of the LM[1-C] multispectral data provided by the SNS sensor.

[0049] In one embodiment, the low-level radiance image rendering MRIR module is configured to separately process each LMi radiance image for a spectral channel i provided by the SNS sensor and produce a full-resolution radiance image for the spectral channel with appropriate visual quality in the luminance domain. Subsequently, the regression TRG module aims to transition from the RDM domain of radiance measurements to the TEDM domain of temperature and emissivity. In one embodiment, the TRG module implements not an iterative algorithm as in the prior art, but a feed-forward neural network to estimate the temperature and emissivity of a scene in real time. The regression TRG module contains only operations at the pixel level, since the temperature and emissivity of each pixel are derived solely from the measured multispectral radiances, the output image from the C+l-value-per-pixel regression TRG module may contain artifacts that are not visually pleasing to the human eye. The refinement modules PRTR and PSTR are designed to correct these artifacts before and / or after processing by the TRG module, ensuring that the final temperature and emissivity images have acceptable visual quality. It should be noted that these artifacts are primarily or predominantly due to inconsistencies or discrepancies between the physical acquisition model and the model used during the initial supervised training.

[0050] Furthermore, in prior art iterative TES methods, the temperature and emissivity estimation for each pixel is performed independently, without considering the values ​​of adjacent pixels. Consequently, if the TES algorithm fails at a pixel, discontinuity artifacts are generated in the output temperature / emissivity images. In contrast, the PRTR and / or PSTR refinement module processes each pixel by taking into account neighboring pixels, in order to eliminate any potential artifacts, thereby improving the visual quality of the temperature / emissivity images.

[0051] Furthermore, prior art iterative TES methods are not conducive to parallel processing and are very time-consuming. In contrast, the proposed processing chain based on a neural network with direct prediction or inference is perfectly compatible with parallel processing and therefore suitable for real-time deployment. If each module is designed compactly, the entire processing chain can be implemented in peripheral devices to perform a first real-time estimation of the temperature / emissivity of a scene. It can also enable high-quality display of the scene, i.e., presenting panchromatic or false-color images, providing clearer information about the scene.

[0052] The proposed processing chain also makes it possible to perform various rendering tasks (e.g., demosaicing, denoising, super-resolution, etc.), while maintaining knowledge of the visual quality of the output temperature / emissivity images. It can therefore be used in different image capture contexts, for example, an IR camera with a repeating pattern of mosaicked multispectral filters applied to the camera's pixel array, or a multispectral camera with low spatial resolution.

[0053] It should also be noted that collecting a dataset of images to perform supervised learning with temperature / emissivity ground truth images of different scenes is costly and practically impractical. Conversely, the proposed processing chain enables absolute thermography from thermal radiation measurements in the absence of such a dataset. This result can be achieved using a semi-supervised learning approach, that is, both supervised (using the physical model and an image database for demosaicing) and self-supervised (using the data from the sensor itself).

[0054] The proposed processing chain also provides a direct output of temperature and emissivity via spectral channel. It can therefore be associated with a decision-making module located at the output of the processing chain to perform real-time applications such as monitoring or anomaly detection.

[0055] According to one embodiment, the processing chain learning procedure comprises three learning steps: STI, ST2, and ST3. The first learning step, STI, is performed only for the regression TRG module. Temperature and emissivity are estimated using a physical model of the SNS sensor. This physical model can be that defined by equations (1) and (2). For this purpose, a training dataset MDD is generated in the form of a table. Alternatively, the physical model is incorporated into a data augmentation scheme to generate the training data incrementally. Each sample of the generated data comprises, as input, C measured radiance values, and as output, a TMP temperature value and C EMS emissivity values ​​[1-C].The TMP temperature and EMSi emissivity for each spectral channel i can be randomly sampled within a given range and then used to calculate the C radiance measurements using the physical model defined by equations (1) and (2) and integrating the spectral response of each filter in a set of C filters. The regression neural network model of the TRG module receives as input the LM[1-C] radiations extracted from the MDD training data and learns to determine the TMP temperature and EMS[1-C] emissivities by minimizing an RLS quality loss function (e.g., the mean squared error function or L2 distance, or the mean absolute error function MAE or LL distance).

[0056] The second learning step ST2 concerns only the MRIR radiance image rendering module. This step is configurable, as it depends on the nature of the multispectral data provided by the SNS sensor and therefore on the processing performed by the MRIR module. According to a first example, the MRIR module performs demosaicing of the received LM[1-C] multispectral images when these images are matrixed in a manner similar to the matrixing observed in images provided by cameras in the visible domain. The LM[1-C] multispectral images are thus provided by a TIR-type SNS sensor (such as a microbolometer) with An array of different filters integrated into the focal plane allows a spectral channel to be associated with each pixel of the sensor. As another example, the MRIR module performs denoising to eliminate noise from the received images. When the input multispectral images are small, the MRIR module can include a super-resolution array to increase image resolution. The MRIR module can also perform realignment between channels to address the spatial shift caused by using a camera with a filter wheel to sequentially capture multispectral images. Furthermore, if radiance is measured by C filters, the MRIR module includes simultaneous reconstruction of C full-frame images. The MRIR module can be trained in a supervised manner with available ground truth annotations.From a set of high-resolution, high-quality images, a transformation (e.g., matrixing, noise addition, or downsampling) is applied to these images to obtain images corresponding to multispectral data provided by the SNS sensor. The MRIR module learns to reverse this transformation to recover the original IMD images by minimizing a loss-of-fidelity function FLS (e.g., of type L1 or L2).

[0057] The third learning step, ST3, involves the PRTR and / or PSTR refinement module, depending on the processing chain configuration, using either one or both of these modules. The IMD' training data used in step ST3 can be IMD' images from the SNS sensor, combined with QLS visual quality data. Following the training of the TRG and MRIR modules, these modules are frozen (i.e., their parameters are fixed). One or both of the PRTR and PSTR refinement modules, which also include neural networks, are incorporated into the MRIR and TRG modules to form the complete processing chain that undergoes the learning process.Unlike the previous steps, a self-supervised learning strategy is implemented to optimize both the PRTR and PSTR refinement modules, since reference temperature and emissivity images are unavailable. The learning objective of this third step is to improve the visual quality (QLS) of the TMP temperature and EMS[1-C] emissivity images provided by the TRG module (e.g., improving smoothness in uniform areas and edge sharpness), while constraining the refined output images to be close to the output of the TRG module without the PRTR and PSTR modules. The visual quality (QLS) is determined by a regularization term during the learning phase at the output of the PSTR module (or, if absent, at the output of the TRG module), in order to guide the module's behavior. PRTR and / or the PSTR module towards the production of pleasing temperature / emissivity images (with improved visual quality).

[0058] For the sake of simplicity and clarity, the temperature and emissivity estimates are separated and performed using two different models. Figure 2 shows an embodiment of the TMPM temperature regression neural network of the TRG module in a case involving a C-channel spectral TES model. The temperature regression neural network comprises several branches BR1-BR5 having fully connected layers (or "dense layers") DRLn, DRLm of n and m units, respectively, including a Rectified Linear Unit (ReLU) activation function to set all negative values ​​to 0. In one exemplary embodiment, n and m are equal to 16 when C equals 4.

[0059] At the beginning of the MRIR module, the minimum MN, maximum MX, and average MY values ​​of the C LMi radiation measurements are calculated. These values ​​are concatenated with the radiation measurements by a CT concatenation operation to obtain a C+3 component VC vector. The VC vector is supplied to three branches BRI, BR2, and BR3, each comprising an n-unit DRLn dense layer and an m-unit DRLm dense layer. In branches BRI and BR3, the m-unit DRLm dense layers are in series with the n-unit DRLn dense layers. Branch BR2 includes a multiplication operator Ml that receives the output of the DRLn dense layer of branch BR2 and the output of the DRLm dense layer of branch BRI. The output of operator Ml is supplied to the input of the DRLm dense layer of branch BR2, as well as to the input of the DRLn dense layer of branch BR4. The dense DRLn layer of the BR4 branch is followed by a sigmoid-type DSGm dense layer with m units.The DSGm dense layer applies the sigmoid activation function (sigmoid(x) = 1 / (1 + exp(-x)), which returns a value very close to zero for values ​​less than -5 and close to one for values ​​greater than 5. The output of the DRLm dense layer of branch BR3 is fed to a one-incrementing XI operator. The output of the DRLm dense layer of branch BR2 is divided by the output of the XI operator using a division DI operator. The output of the DI operator is fed to the DRLn dense layer of branch BR5. The DRLn dense layer of branch BR5 is followed by a DRLm dense layer. The output of the DSGm layer of branch BR4 and the output of the DRLm dense layer of branch BR5 are fed to the input of an addition Al operator. The output of the Al operator is fed to the input of a one-unit linear dense layer DL1, which provides a temperature image TMP.The loss function to be minimized during the training of the neural network in Figure 2 can be the mean absolute error function (Ll).

[0060] Figure 3 represents an embodiment of the EMSM emissivity regression neural network of the TRG module, also in the case implementing A C-channel spectral TES model. This neural network comprises two branches, BRU and BR12. Each branch includes two DRLnl dense layers, in series, with n1 and n2 units respectively, each DRLnl, DRLn2 dense layer including a ReLU-type activation function. The BR12 branch includes a TMPM temperature regression model as a layer, in series with one DRLnl layer and one DRLn2 layer. The output of the DRLn2 layer of the BR 12 branch is fed to an X2 increment operator of 1 whose output forms the output of the BR 12 branch. The output of the DRLn2 layer of the BRI 1 branch and the output of the BR 12 branch are combined by a D2 split operator, before being converted into C dimensions of EMS emissivity by a dense DCRL layer of C units including a clipped ReLU activation function to keep the emissivity values ​​in the appropriate range (typically between 0.1 and 1.0).The DCRL dense layer provides the C-component EMS[1-C] emissivity image. The mean absolute error function L1 can be used as the fidelity loss function of the emissivity model.

[0061] The TRG module learning procedure (STI step) comprises two steps, namely: firstly, the training of the temperature neural network, then the training of the emissivity neural network with the TMPM temperature regression model, initialized from the trained temperature neural network.

[0062] As previously mentioned, one possible scenario involves measuring multispectral radiance using a TIR bolometer-type SNS sensor with a filter in its focal plane having a mosaic pattern of 2 x 2 elements, each element covering one pixel of the sensor. Thus, if the images provided by the SNS sensor have H x W pixels, the pixels for a spectral channel are spaced one pixel apart in both directions of the image. The images for each of the spectral channels i therefore have H / 2 x W / 2 pixels. In this case, the MRIR module performs a demosaicing operation to provide full-frame multispectral images of size H x W for each channel i.Due to the complete absence of strong physical assumptions (particularly regarding inter-channel correlation) and the spatial misalignment between the available TIR filters, demosaicing can only be performed independently of the channels; that is, each spectral channel is reconstructed without using information from the other channels. This problem is equivalent to that of image retouching, in which three missing pixel values ​​must be determined for each 2x2 pixel window in each of the LM radiance images provided by the SNS sensor.

[0063] Figure 4 shows a CSDM demosaicing network, according to one embodiment. This demosaicing network comprises C (= 4) processing channels to reconstruct a radiance image in each of the spectral channels in a manner independent, from a matrixed TIR image. The input image (from the sensor), size H x W, is represented in Figure 4 by a 2 x 2 pixel IWI image window containing the pixels corresponding to the four spectral channels. The input image of the MRIR module is duplicated into four H x W images, each containing only the pixels related to a respective channel of the input image. These four matrixed images are each represented by a 2 x 2 pixel window DW1, DW2, DW3, DW4, each containing one respective pixel from the IWI window, the other three pixels being undetermined. Each of the images represented by one of the DW1-DW4 windows is processed separately by the CSDM network, which implements a unique demosaicing model, to determine the undetermined pixels of each of the input images to the CSDM network.The four images thus completed, corresponding respectively to the four channels, and symbolized by the spectral windows RW1, RW2, RW3, RW4, are then processed by a CCT concatenation operator to obtain a complete DTC frame of size H x W x C pixels of the multispectral radiation.

[0064] Figure 5 shows in more detail how the raster image of each channel i can be super-resolved in a coloring manner. The raster image, symbolized in Figure 5 by a 4 x 4 pixel DDWI window, is processed by a DIPF deep retouching module to obtain a full-frame intermediate image, symbolized by a 4 x 4 pixel IIM window, in which all pixels are determined. The intermediate image is then combined by a multiplication operator M3 with a binary mask image BIM, and then combined by an addition operator A3 with the raster image in such a way as to retain the determined pixels of the input raster image (pixels labeled 11, 12, 13, 14 in Figure 5). Thus, the determined pixels of the DDWI matrix image are assigned to the same positions in the intermediate IIM image while preserving the other pixels of the IIM image to obtain a reconstructed channel image symbolized by the RIM window.

[0065] An example of the DIPF module architecture is shown in Figure 6. The DIPF module comprises a fully convolutional neural network. To mitigate boundary artifacts, before each convolution, one pixel on each spatial side of the data is mirrored. Then, a valid convolution is performed to maintain the data dimensionality unchanged. The expressiveness of the model can be increased by using residual connections. At the end of the model, a local rearrangement of the channel pixel data is used to double the horizontal and vertical spatial dimensions and obtain a full-frame image (i.e., at the sensor's spatial resolution). For this purpose, the DIPF function includes an SSMP subsampling module, an LKRn layer receiving the output image from the SSMP module, and three serial RBLK modules receiving The output of the LKRn layer. The SSMP module receives the image symbolized by the DDWI window, corresponding to spectral channel i, and removes the undefined pixels from the image to obtain an image symbolized by a 2 x 2 pixel W2 window containing only the defined pixels in the DDWI image. The DIPF module also includes an A4 addition operator combining the output of the LKRn layer with the output of the series of three RBLK modules. The output of the addition operator is provided at the input of another LKRp layer in series with a DPSP data reorganization module and an RPRp layer. The DPSP module reorganizes the pixel data distributed in H x W x D arrays using a 2 x 2 pixel window, merging the data planes along the depth D into D / 4 planes of size 2H x 2W.

[0066] The LKRn module applies a one-pixel image enlargement function, then a valid 3 x 3 convolution with n units, and finally, a "Leaky ReLU" type activation function that is transparent for positive values ​​and attenuates negative values ​​with a linear function. The one-pixel enlargement function of the LKRn module creates a reflected frame one pixel wide around a window, for example, 4 x 4 pixels. Thus, the left border of the enlarged window receives the pixels of the second column of the initial window, the right border of the enlarged window receives the pixels of the second-to-last column of the initial window, the top border of the enlarged window receives the pixels of the second row of the initial window, and the bottom border of the enlarged window receives the pixels of the second-to-last row of the initial window.Furthermore, the pixel in the upper left corner of the enlarged window receives the pixel in the second row and second column of the initial window, the pixel in the upper right corner receives the pixel in the second row and second-to-last column of the initial window, the pixel in the lower left corner of the enlarged window receives the pixel in the second-to-last row and second column of the initial window, and the pixel in the lower right corner receives the pixel in the second-to-last row and second-to-last column of the initial window. The RPRp layers apply a one-pixel image upscaling function, a valid 3x3 convolution with p units, and a ReLU-type activation function. In one example embodiment, n = 16 and p = 64.

[0067] Figure 7 shows an example embodiment of the RBLK module. The RBLK module comprises an LKRn layer in series with an LKRq layer, the input of the LKRn module being added to the output of the LKRq module by an addition operator A5. The output of the A5 operator constitutes the output of the RBLK module. In one example embodiment, q = 8.

[0068] According to one embodiment, the MRIR module is trained using a set of single-channel infrared images corresponding to the IMD image set in the Figure 1. To this end, small 128 x 128 pixels are randomly extracted from each image in the set. Then, one pixel from each 2x2 window in each thumbnail is extracted to generate the DDWI rasterized image as input to the MRIR module. The loss of fidelity function used can be the mean absolute error function (L1).

[0069] Figure 8 shows the architecture of the PRTR refinement module, according to an exemplary embodiment. In this exemplary embodiment, the PRTR module comprises a feature branch BR21 for improving the quality of the DTC image, and an attention branch BR22, with branches BR21 and BR22 receiving the DTC image. Branch BR22 adaptively combines the improved image with the DTC image to ensure that the output images preserve the original overall structure. Branch BR21 comprises an LKRn layer in series with three RBLK modules (Figure 7) in series. The output of the LKRn module is combined by an A6 addition operator with the output of the third RBLK module in the series of RBLK modules. The output of the A6 addition operator is fed to an RPRr layer with r units, the output of which forms the output of branch BR21. The output of the RPRr layer is fed to an M6 multiplication operator.

[0070] Branch BR21 comprises an LKRn layer in series with an RPSG layer. The RPSG layer applies a valid 3 x 3 convolution to r units having a sigmoid activation function. The output of the RPSG layer is fed to the input of an X6 operator applying f(x) = 1 - x and an M7 multiplication operator. The output of the X6 operator is fed to the M6 ​​multiplication operator. An A7 addition operator combines the output of the M6 ​​operator with the output of the M7 operator and provides the enhanced DTCC image. The DTCC image has a size of H x W pixels with B channels, B not necessarily being equal to the initial number C of spectral channels. In one embodiment as described in Figure 8, r = 4 and B = C.

[0071] The PRTR module can be trained with images acquired from a matrix filter bolometric sensor having C filters corresponding to C spectral channels, used to train the TES regression TRG module.

[0072] As mentioned previously, the processing chain can contain both the PRTR and PSTR refinement modules, or only one of them. In an embodiment illustrated in Figure 9, only one PRTR refinement module is implemented. In this embodiment, the processing chain comprises a TNW processing chain without refinement, designated as the "teacher network," and a complete SNW processing chain with a PRTR refinement module, designated as the "student network." The TNW and SNW networks share the same MRIR demosaicing and TES regression TRG modules trained in the preceding steps. During the training of the SNW student array, the parameters of the MRIR and TRG modules can be fixed or adjusted by regularization to maintain the outputs of these modules close to their values ​​prior to this adjustment. The SNW and TNW arrays take as input the IWI matrix image captured by the camera, then reconstruct four full-frame radiation images RW1-RW4 using the MRIR module (performing demosaicing). These arrays produce C+1 full-frame images including a TMP temperature image and C EMS emissivity images.

[0073] According to one embodiment, the PSTR refinement module has an architecture identical or similar to that shown in Figure 8.

[0074] The main objective of the PRTR and PSTR refinement modules is to improve the quality of the TMP thermometric image, for example by refining edges while ensuring a uniform intensity level in a region originating from the same object in the scene. To this end, Total Variation (TV) regularization is applied to the output Ys images of the SNW student array (LTV = IVYsl). This LTV regularization term is used as a quality metric to drive the optimization of the PRTR and / or PSTR refinement module. Furthermore, processing may be implemented to ensure that the refinement module does not alter the overall detail of the output image, i.e., that the Ys output remains close to that of the TNW teacher array.Therefore, a loss of similarity function (e.g., Ll) can also be used to constrain the behavior of the PRTR and / or PSTR module to not remove global image structures compared to unrefined images. This similarity function can be focused on specific regions and images chosen either by an operator or automatically by an algorithmic process. According to an example embodiment in which the MRIR and TRG modules are fixed, the objective learning function of the PRTR and / or PSTR module is as follows:

[0075] L = a Ltv+ (1 - a) Lsim (3)

[0076] in which a = 9.5x107 is a hyperparameter controlling the effect of the refinement module PRTR, and Lsim is a similarity or fidelity cost function between the output image and the input image of the refinement module. If the MRIR and TRG modules need to be adjusted, additional similarity losses can be adopted between the outputs of these modules of the teacher TNW and student SNW networks. According to another embodiment, the distance (e.g., L2) between the adjusted parameters of the student SNW network and the original parameters (of the teacher TNW network) is regularized.

[0077] Figure 12 shows the SNS multispectral radiance image sensor, according to one embodiment. The SNS sensor comprises an SNA array of elements Radiance-sensing element (RSE) detectors, for example of the microbolometer type, are mounted on top of an FTA array of spectral filter elements (FTEs) arranged in the focal plane of the SNA array. Some FTEs are not shown in Figure 12 for clarity, to better illustrate the SNA array and its radiance-sensing RSE elements. In the example in [Fig. 12], the FTA array comprises four different types of FTE filter elements arranged in a mosaic pattern of four filter elements, corresponding to the IWI window in Figure 4, to obtain radiance images in four spectral channels. The respective bandwidths of the four types of FTE filter elements may partially overlap or be completely separate. The RSEs are sensitive to infrared radiation, for example, in the wavelength band extending from 7.5 pm to 14 pm.

[0078] Figure 13 shows components of a PRD image acquisition and processing device. The PRD device comprises at least the PRC processor, and functionally coupled to the processor, the SNS sensor, MEM memories, and NIT circuits for wired and / or wireless transmission or communication. The MEM memories store an operating system, and in particular, an application dedicated to processing images from the SNS sensor. To this end, the application implements the processing chain described above, including the MRIR, TRD, and PRTR and / or PSTR modules. The NIT communication circuits allow the PRC processor to be functionally coupled to an NT communication network such as the Internet, in particular to transmit the images output from the SNS sensor and / or the images provided by the dedicated application.

[0079] It will be evident to those skilled in the art that the present invention is susceptible to various embodiments and applications. In particular, the invention is not limited to the module architectures described above. Thus, the main function of the MRIR module is to provide full-resolution multispectral radiance images usable in an absolute thermography process. Depending on the nature of the camera, the MRIR module can play different roles such as demosaicing, denoising, super-resolution, realignment, etc., in order to convert the raw sensor measurements into suitable full-resolution multispectral images. These operations can be performed channel by channel (i.e., without information exchange between the spectral channels), or by correlating the channels with each other (i.e., similarly to RGB images).

[0080] The main function of the TES regression TRG module is to provide an estimate of the TMP temperature and EMS emissivity for each pixel of the input image. The emissivity can be estimated based on the assumption of a constant value in each spectral channel, as previously described.

[0081] According to one embodiment, the emissivity is estimated based on the more general assumption that it varies with wavelength and that it can be decomposed into a linear combination of K basis functions gj, in the following manner: 100821

[0083] Under this assumption, the radiance measured for each spectral channel i extending between wavelengths Xmax and Xmax can be calculated using the following equation:

[0084] J [£(X)RBg(T,X)+(l-£(X))RBB(Te,X))]fj(X)dX

[0085] where E(X) is the emissivity of the target object at wavelength X, λ(X) is the spectral response of a filter covering the wavelength band extending between wavelengths Xmin and Xmax, and RBb is the spectral radiance density emitted directly by a black body at temperature T. The spectral radiance density RBB can be calculated using the following Planck equation:

[0086] Rbb(T,X)=j^(6)

[0087] with h = 6.626 x 10³⁴ m².kg / s, c = 3 x 10⁸ m / s, k = 1.38 x 10²³ m².kg / s² K

[0088] Equations (4), (5), (6) are used as a physical sensor model for generate the MDD training dataset. Instead of providing an EMSi emissivity image per spectral channel i, the method provides an image of the K coefficients aj of the spectral emissivity response curve.

[0089] Furthermore, according to another embodiment, the TES module is configured to also provide a confidence level for each predicted value, or even a range of possible values ​​for each estimate. With this information, the refinement module (in particular the PSTR module) can refine the image more easily to produce pleasing thermographic images.

[0090] As previously described, the learning procedure comprises three steps that can be performed simultaneously, without necessarily requiring sequencing in terms of learning from one step to the next. In the case of learning the MRIR and TRG modules, other supervised fidelity loss functions can be used, such as the mean absolute error (L1), the mean squared error (L2), etc. In particular, quality regularization can be introduced during the MRIR module training. In the case of the refinement modules, in addition to the total variation (TV), existing BIQA measurements can be considered as an alternative regularization of image quality. Furthermore, the similarity loss can be any supervised fidelity loss. The modules MRIR and TRG can be fixed or adjusted by regularization during the training of the PRTR and PSTR refinement modules. Alternatively, adaptive loss with TV regularization applied only to regions around edges and similarity loss applied only to uniform areas can also be used. An adversarial framework can also be considered for training the PRTR and PSTR refinement modules. This training can involve using a discriminator in parallel with the proposed processing chain (considered the generator in the domain semantics). With a database of real, high-quality temperature / emissivity images available, the processing chain creates images, while the discriminator evaluates the quality of these images (classifying them as real and of high visual quality).The two networks can train in a loop, which allows their respective performances to improve.

[0091] Furthermore, as previously described, the processing chain comprises three elements: the MRIR module, the TES regression TRG module, and the refinement processing. The refinement processing can be performed by two refinement modules, PRTR and PSTR, or by a single refinement module, to improve the visual quality of the TMP thermal image and the EMS emissivity images. However, other modules dedicated to other applications can be integrated. For example, additional processing can be provided for estimating a panchromatic radiance image or a false-color thermal image from the multispectral images.Thus, Figure 10 represents a processing chain that differs from that of Figure 1 in that it includes a panchromatic estimation PCE module producing a panchromatic radiance PIM image, and a false-color FCIM thermal estimation FCE module receiving the pre-processed image output from the PRTR module. According to another embodiment, the panchromatic estimation PCE module is located at the output of the TRG or PSTR module to provide false-color temperature and / or emissivity images.

[0092] Each of the MRIR, TRG, PRTR, and PSTR modules described above can adopt several types of neural network architectures. In the case of the TES regression TRG module, only per-pixel calculations are considered, since the temperature / emissivity estimation at each spatial position is performed independently. Therefore, the model used can be built solely on 1x1 convolutions, i.e., fully connected layers. For the MRIR, PRTR, and PSTR modules, a convolutional neural network can be implemented with a different kernel size, greater than 1x1. Furthermore, if the multispectral measurements are in the form of a video (i.e., as a continuous acquisition of images over time), other structures of models can be used for MRIR, PRTR and PSTR modules, to obtain spatiotemporal features, such as 3D convolutions, or recurrent neural networks (RNNs).

[0093] The present invention paves the way for estimating TMP temperature and EMS emissivity images of a scene, using a neural network architecture suitable for real-time deployment on a wide variety of hardware platforms. It provides a unified framework for estimating the temperature and emissivity of each pixel in a scene while taking into account the visual quality of the resulting temperature and emissivity images. It offers a model learning strategy compatible with the current lack of a suitable training database, thus enabling fully supervised learning. Furthermore, the processing chain is versatile, being independent of the acquisition method for the multispectral radiance measurements.

[0094] Furthermore, the use of TIR images in addition to visible RGB images is now a popular topic, as it offers more comprehensive perception capabilities regardless of lighting or weather conditions. The resulting TMP temperature and EMS emissivity images can be fused with images from a visible image sensor to improve the algorithmic performance of the inference task, particularly when the visible image is of poor quality due to adverse weather or lighting conditions. Besides the direct use of the processing chain described above to obtain a real-time absolute thermographic image of a scene, other applications can be considered in the fields of monitoring, security, or anomaly detection, based on temperature and emissivity images.To this end, it is possible to add in series to the previously described processing chain, a suitable inference module (i.e., one that allows understanding of the scene, and whose output can typically correspond to a pixel-based segmentation of the image content), with the possibility of training this module separately or jointly with the PRTR, PSTR refinement modules.

[0095] It may also be envisaged to integrate other inference applications derived from the pre / post-TES regression model (Figure 11). Thus, Figure 11 represents a processing chain that differs from that of Figure 1 in that it includes a PMD perception module that can receive images from the PRTR or PSTR module and provides an INFD inference decision. For example, the PMD module can be a neural network module configured to detect areas of anomalies, estimate materials in the scene by pixel, or detect a certain event of interest. References Cited

[0096] [1] A. Gillespie, S. Rokugawa, T. Matsunaga, JS Cothern, S. Hook, and AB Kahle, “A temperature and emissivity separation algorithm for Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) images,” IEEE Trans. Geosci. Remote Sens., vol. 36, n° 4, p. 1113-1126, juill. 1998, doi: 10.1109 / 36.700995.

[0097] [2] A. Barducci, D. Guzzi, C. Lastri, P. Marcoionni, V. Nardino, et I. Pippi, « Emissivity and Température Assessment Using a Maximum Entropy Estimator: Structure and Performance of the MaxEnTES Algorithm », IEEE Trans. Geosci. Remote Sens., vol. 53, n° 2, p. 738-751, févr. 2015, doi: 10.1109 / TGRS.2014.2327218.

[0098] [3] F. Aires, A. Chédin, N. A. Scott, et W. B. Rossow, « A Regularized Neural Net Approach for Retrieval of Atmospheric and Surface Températures with the IASI Instrument », J. Appl. Meteorol. Climatol., vol. 41, n° 2, p. 144-159, févr. 2002, doi: 10.1175 / 1520-0450(2002)041<0144: ARNNAF>2.0.CO;2.

[0099] [4] M. Chen, L. Ni, X. Jiang, Z. Li, et H. Wu, « Retrieval of Atmospheric and Land Surface Parameters from Satellite-Based Thermal Infrared Hyperspectral Data Using an Artificial Neural Network Technique », in IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia: IEEE, juill. 2018, p. 2745-2748. doi: 10.1109 / IGARSS.2018.8518131.

[0100] [5] X. Ye et al., « Simultaneous Estimation of Land Surface and Atmospheric Parameters From Thermal Hyperspectral Data Using a LSTM-CNN Combined Deep Neural Network », IEEE Geosci. Remote Sens. Lett., vol. 19, p. 1-5, 2022, doi: 10.1109 / LGRS.2021.3104501.

[0101] [6] F. Bao et al., « Heat-assisted détection and ranging », Nature, vol. 619, n° 7971, Art. n° 7971, juill. 2023, doi: 10.1038 / s41586-023-06174-6.

[0102] [7] V. Saragadam, A. Dave, A. Veeraraghavan, et R. G. Baraniuk, « Thermal Image Processing via Physics-Inspired Deep Networks », in 20211EEE / CVF International Conférence on Computer Vision Workshops (1CCVW), Montreal, BC, Canada: IEEE, oct. 2021, p. 4040-4048. doi: 10.1109 / ICCVW54120.2021.00451.

[0103] [8] P. Shyam et H. Yoo, « Lightweight Thermal Super-Resolution and Object Détection for Robust Perception in Adverse Weather Conditions ».

[0104] [9] P. Yang, H. Wu, L. Cheng, et S. Luo, « Infrared image denoising via adversarial leaming with multi-level feature attention network », Infrared Phys. Technol., vol. 128, p. 104527, janv. 2023, doi: 10.1016 / j.infrared.2022.104527.

[0105]

[10] S. Agaian, M. Roopaei, et D. Akopian, « Thermal-image quality measurements », in 2014 IEEE International Conférence on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy: IEEE, mai 2014, p. 2779-2783. doi: 10.1109 / ICASSP.2014.6854106.

[0106]

[11] M. Teutsch, S. Sedelmaier, S. Moosbauer, G. Eilertsen, et T. Walter, « An Evaluation of Objective Image Quality Assessment for Thermal Infrared Video Tone Mapping », in 2020IEEE / CVF Conférence on Computer Vision and Pattern Récognition Workshops (CVPRW), Seattle, WA, USA: IEEE, juin 2020, p. 488-497. doi: 10.1109 / CVPRW50498.2020.00062.

[0107]

[12] F. Erlenbusch et al., « Thermal Infrared Single Image Dehazing and Blind Image Quality Assessment », in 2023 IEEE / CVF Conférence on Computer Vision and Pattern Récognition Workshops (CVPRW), Vancouver, BC, Canada: IEEE, juin 2023, p. 459-469. doi: 10.1109 / CVPRW59228.2023.00052.

Claims

Demands

1. 1. A method for processing infrared thermal images, IR, comprising steps of: generating training data (MDD) from a physical model of a bolometric sensor comprising several spectral channels, the physical model providing for each spectral channel a radiance measurement as a function of temperature and emissivity values ​​in the spectral channel or as a function of coefficients of a linear combination of several spectral response curves; training a regression module (TRD) comprising a forward-predicting neural network (TMPM) to estimate a temperature image (TMP) and an emissivity image (EMSi) for each spectral channel (i) or coefficient, from the training data;train an image rendering module (MRIR) configured to produce a full-resolution multispectral radiance image of appropriate visual quality in a luminance domain, from a database (IMD) of real images; train a processing chain comprising the rendering module, the regression module, and at least one refinement module (PRTR, PSTR) disposed between the rendering module and the regression module and / or at the output of the regression module, the rendering module and the regression module being frozen or adjusted by regularization during training; acquire by a processor (PRC), a radiance image (LM[1-C]) provided by a multispectral bolometric sensor (SNS), the radiance image comprising for each pixel a radiance value for each spectral channel or each coefficient;and process the image acquired by the driven processing chain, implemented by the processor, to obtain a temperature image and an emissivity image per spectral channel or per coefficient.

2. 2. A method according to claim 1, wherein the regression module (TRG) comprises a temperature image estimation (TMP) neural network which is trained and then used to train an emissivity image estimation (EMSi) neural network for each spectral channel or for each coefficient.

3. 3. Method according to claim 1 or 2, wherein the processing chain comprises a first refinement module (PRTR) disposed between the rendering module (MRIR) and the regression module (TRG) and a second refinement module (PSTR) disposed at the output of the regression module, the first and second refinement modules being driven at the same time during the training of the processing chain.

4. 4. A method according to any one of claims 1 to 3, wherein the radiance image rendering module (MRIR) (LM[1-C]) is configured to separately process each radiance image (LMi) for a spectral channel (i) supplied by the sensor (SNS) and produce a full-resolution radiance image for the spectral channel with appropriate visual quality in the luminance domain.

5. 5. A method according to any one of claims 1 to 4, wherein the refinement module (PRTR, PSTR) is configured to process each pixel of the radiance images (LM[1-C]) taking into account neighboring pixels, to remove possible artifacts, and improve the visual quality of the temperature / emissivity images.

6. 6. A method according to any one of claims 1 to 5, wherein the image rendering module (MRIR) performs at least one of the following processing: a demosaicing process, a denoising process, a resolution upscaling process, and a spectral channel realignment process.

7. 7. A method according to any one of claims 1 to 6, wherein the processing chain drive is carried out in a self-supervised manner or without using reference temperature and emissivity images.

8. 8. A method according to any one of claims 1 to 7, wherein the training of the processing chain is carried out using a fidelity or loss of similarity function to constrain the behavior of the refinement module (PRTR, PSTR) not to remove global structures in the images compared to unrefined images provided by the regression module (TRG).

9. 9. A method according to any one of claims 1 to 8, wherein the refinement module (PRTR, PSTR) is configured to perform smoothing and / or denoising processing in uniform areas and edge sharpness of temperature (TMP) and emissivity (EMS[1-C]) images provided by the regression module (TRG), by constraining the output images of the processing chain to approach the output images of the regression module.

10. 10. A method according to any one of claims 1 to 9, wherein the processing chain comprises: a panchromatic estimation processing (PCE) producing a panchromatic radiance image (PIM) from an input or output image of the regression module (TRG), and / or a false-color thermal image estimation (FCIM) processing receiving the input and / or output image of the regression module (TRG), and / or output image of the refinement module (PSTR).

11. 11. A method according to any one of claims 1 to 10, comprising a step of fusing temperature image (TMP) and / or emissivity images (EMS[1-C]) with images from an image sensor in the visible range.

12. 12. A method according to any one of claims 1 to 11, wherein the processing chain comprises a perception module (PMD) receiving images from the refinement module (PRTR, PSTR), and which provides an inference decision (INFD).

13. 13. Device for generating multispectral temperature and emissivity images, comprising: a multispectral radiance sensor (SNS), and a processor (PRC) configured to implement the steps of the process according to any one of claims 1 to 12.

14. 14. Device according to claim 13, wherein the sensor (SNS) is of the infrared thermal bolometer type, TIR, comprising a filter disposed in a focal plane of the sensor and having a mosaic pattern of 2 x 2 filter elements, each filter element of the pattern corresponding respectively to one of the spectral channels.

15. 15. A computer-readable recording medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 12.

Images