An infrared temperature measurement method and system based on pixel offset super-resolution

Abstract

The application provides an infrared temperature measurement method and system based on pixel offset super-resolution, which comprises the following steps: constructing an infrared Raw image sequence comprising N infrared Raw images; extracting multi-scale features of each infrared Raw image based on an L-layer feature pyramid structure, and aligning each to-be-featured aligned image with the first layer features of a reference image based on the multi-scale features; fusing each first layer aligned feature with the reference image respectively to obtain fused features; processing the fused features based on an image super-resolution reconstruction network to output a high-resolution image; and obtaining a target temperature based on the high-resolution image. The application can simulate pixel offset situations occurring in different scenarios, align multiple low-resolution images through image registration, and finally fuse the low-resolution images to generate a high-resolution image with higher spatial resolution based on an image super-resolution reconstruction network, so as to greatly improve the temperature measurement accuracy of the target.

Description

Technical Field

[0001] This invention relates to the field of infrared imaging technology, and in particular to an infrared temperature measurement method and system based on pixel offset super-resolution. Background Technology

[0002] The accuracy and spatial resolution of infrared thermal imaging temperature measurement technology are core performance indicators. Currently, the main technical solutions for improving the accuracy and spatial resolution of infrared temperature measurement are based on hardware upgrades and software algorithm optimization, for example:

[0003] 1. Use infrared detectors with more pixels, higher focal plane array resolution, and smaller pixel size. However, such infrared detectors are expensive and require matching high-resolution, large-aperture (smaller F-number) infrared lenses, which leads to an exponential increase in cost and makes it difficult to promote them widely.

[0004] 2. Enhancing infrared image contrast through image processing methods such as feature map fusion highlights image details, thereby improving temperature measurement accuracy. However, this approach primarily focuses on enhancing image details from a subjective visual perspective. Nevertheless, it is limited by insufficient spatial resolution, the fundamental cause of temperature measurement errors. This means that the signal perceived by a single pixel is the average of the radiated energy from multiple objects at different temperatures (mixed pixel effect). Software algorithms cannot artificially create the lost high-frequency spatial information, resulting in limited improvement in the accuracy of temperature measurement results. Summary of the Invention

[0005] The purpose of this invention is to provide an infrared temperature measurement method and system based on pixel offset super-resolution, which can simulate pixel offset situations occurring in different scenarios, align multiple low-resolution images through image registration, and finally reconstruct the image based on an image super-resolution reconstruction network to fuse multiple aligned low-resolution raw images to generate a high-resolution image with higher spatial resolution, and calculate the target temperature based on a temperature measurement model to significantly improve the temperature measurement accuracy of targets (especially small targets and distant targets).

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] On the one hand, an infrared thermometry method based on pixel offset super-resolution is provided, which includes the following steps:

[0008] Construct an infrared raw image sequence containing N frames of infrared raw images;

[0009] Based on the L-layer feature pyramid structure, multi-scale features of each infrared raw image in the infrared raw image sequence are extracted, and each image to be aligned with the first layer features of the reference image is aligned with the multi-scale features to obtain the first layer alignment features of each image to be aligned.

[0010] Each first-layer alignment feature is fused with a reference image to obtain fused features;

[0011] The fusion features are processed based on the image super-resolution reconstruction network to output a high-resolution image;

[0012] Additionally, the target temperature is obtained based on high-resolution images.

[0013] On the other hand, an infrared temperature measurement system is also provided, which includes:

[0014] The feature extraction module extracts multi-scale features F of each infrared raw image in the infrared raw image sequence based on an L-layer feature pyramid structure.

[0015] The feature alignment module aligns each image to be aligned with the first layer features of the reference image based on multi-scale features F.

[0016] The feature fusion module is used to fuse each first-layer aligned feature with the reference image to obtain fused features;

[0017] The image super-resolution module processes the fused features based on the image super-resolution reconstruction network to output a high-resolution image;

[0018] And, a target temperature measurement module, which acquires the target temperature based on the reconstructed high-resolution image.

[0019] In summary, the present invention has the following advantages compared with the prior art:

[0020] This invention can simulate pixel shifts occurring in different scenarios through two image sequence construction methods, thereby enhancing the accuracy of infrared temperature measurement results in different scenarios. Simultaneously, multiple low-resolution images are aligned through image registration, and finally, image reconstruction is performed based on an image super-resolution reconstruction network. This allows for the fusion of multiple aligned low-resolution raw images to generate a high-resolution image with higher spatial resolution. The target temperature is then calculated based on a temperature measurement model, significantly improving the temperature measurement accuracy of targets (especially small and distant targets).

[0021] Furthermore, the implementation of the technical solution of the present invention is not limited by the size of the infrared detector pixel. Therefore, it will not cause problems such as the averaging of temperature information in a single pixel and inaccurate temperature measurement results for small or distant targets due to excessively large pixel size or insufficient number of pixels. It can significantly improve the temperature measurement performance of existing infrared temperature measurement equipment without replacing high-cost, large-array infrared detectors. Attached Figure Description

[0022] Figure 1This is a flowchart of the infrared temperature measurement method in this invention;

[0023] Figure 2 This refers to a low-resolution infrared raw image in the infrared raw image sequence of this invention;

[0024] Figure 3 This is a schematic diagram illustrating the acquisition of low-resolution infrared raw images through a first image sequence construction method in this invention;

[0025] Figure 4 This is a schematic diagram illustrating the acquisition of low-resolution infrared raw images through a second image sequence construction method in this invention;

[0026] Figure 5 These are the row cumulative curves and column cumulative curves after shfitX and shfitY are applied to the image in this invention.

[0027] Figure 6 This is a schematic diagram of the high-resolution image T_HR output process in this invention;

[0028] Figure 7 The target image output by existing technology and the high-resolution image T_HR output by this invention;

[0029] Figure 8 for Figure 7 A magnified image of point A in the middle;

[0030] Figure 9 This is a schematic diagram of the infrared temperature measurement system in this invention. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] Figure 1 As shown, this embodiment provides an infrared temperature measurement method based on pixel offset super-resolution, which includes the following steps:

[0034] S1. Acquire N frames of low-resolution infrared Raw images (e.g.) Figure 2 (as shown), and construct an infrared Raw image sequence;

[0035] In this embodiment, the low-resolution infrared raw image is Y16 data, and the infrared raw image sequence is constructed using either a first image sequence construction method or a second image sequence construction method. The first image sequence construction method includes:

[0036] like Figure 3 As shown, a high-precision micro-displacement driving device (such as piezoelectric ceramics) drives the optical components of an infrared imaging device to move m times along the X-axis and n times along the Y-axis within the array area, thereby achieving sub-pixel-level precision pixel offset control. The optical components are infrared detectors (i.e.,...) Figure 3 The "sensor" in the text) or infrared camera (i.e. Figure 3 The “Lens” in the text, and the X-direction offset shfitX and the X-direction offset shfitY that move along the X direction each time can be the same or different, and the XY plane is perpendicular to the optical axis direction P.

[0037] Each time the movement is completed (i.e., one movement along the X direction or one movement along the Y direction), one frame of infrared Raw image is acquired, until N frames of infrared Raw images are acquired.

[0038] Construct an infrared raw image sequence T_align containing N frames of infrared raw images;

[0039] The second image sequence construction method includes:

[0040] like Figure 4 As shown, the infrared imaging device is driven to randomly shake several times by hand or device drive, and one frame of infrared raw image is acquired during each shaking process, until N frames of infrared raw images are acquired.

[0041] The first frame of infrared raw image is used as the reference image T. ref And calculate the reference image T respectively. ref The accumulated row value T_rows ref Column accumulated values ​​T_cols ref And the need for reference image T ref The accumulated row values ​​T_rows of the k-th frame of the infrared raw image to be aligned. k Column accumulated values ​​T_cols k The infrared raw images to be aligned are the remaining N-1 infrared raw images excluding the first infrared raw image.

[0042] Based on the X-axis offset shfitX and the Y-axis offset shfitY, the infrared raw image to be aligned in the k-th frame is translated to align the infrared raw image to be aligned with the reference image T. refAlignment is performed, and this step is repeated until all the remaining N-1 frames of infrared raw images to be aligned are aligned with the reference image T. ref Alignment, to reduce the difficulty of pixel-level alignment in later stages, specifically includes the following steps:

[0043] Iterate through the range of translation values ​​[-ShfitMax, ShfitMax] to obtain the value of the current Y-direction translation mY, where mY∈[-ShfitMax, ShfitMax], and -ShfitMax and ShfitMax are the preset minimum and maximum translation values, respectively.

[0044] After each time the current Y-axis translation value mY is obtained, the row accumulation value T_rows of the infrared Raw image to be aligned in the k-th frame is incremented. k The row is shifted according to the current Y-axis shift amount mY, and the row accumulation value T_rows is calculated after each shift. k _m k mY < 0 indicates upward translation, mY > 0 indicates downward translation; -ShfitMax, ShfitMax, and mY can all be counted in pixels;

[0045] Calculate the cumulative row count T_rows after each shift. k _m k Compared with reference image T ref The accumulated row value T_rows ref The difference between them is detaY, and the Y-direction translation mY corresponding to the minimum value of the difference detaY is determined as the Y-direction offset shfitY;

[0046] Iterate through the range of translation values ​​[-ShfitMax,ShfitMax] to obtain the value of the current X-direction translation mX, where mX∈[-ShfitMax,ShfitMax].

[0047] After each time the current X-axis translation value mX is obtained, the column accumulation value T_cols of the infrared Raw image to be aligned in the k-th frame is incremented. k The column is shifted according to the current X-axis shift amount mX, and the cumulative column value T_cols is calculated after each shift. k _m k mX < 0 indicates a leftward shift, and mX > 0 indicates a rightward shift; mX can be counted in pixels.

[0048] Calculate the cumulative column value T_cols after each translation. k _m k Compared with reference image T ref The cumulative column values ​​T_cols refThe difference between them is detaX, and the X-direction displacement mX corresponding to the minimum value of the difference detaX is determined as the X-direction offset shfitX;

[0049] Each infrared raw image to be aligned, except for the first infrared raw image, is translated by an offset of shfitX in the X direction and an offset of shfitY in the Y direction, so that each infrared raw image to be aligned is aligned with the first infrared raw image.

[0050] For example, Figure 5 This shows the infrared raw image to be aligned in the k-th frame (i.e., Figure 5 The "k-th frame image" in the image is set according to shfitX=-6 (i.e., Figure 5 In the example, "horizontalShift=-6") and shfitY=-1 ​​(i.e. Figure 5 The row and column cumulative curves after shifting "verticalShift=-1" in the text;

[0051] After image alignment is completed, an infrared raw image sequence T_align containing N frames of infrared raw images can be constructed. The infrared raw image sequence T_align includes the first frame of infrared raw image and all infrared raw images aligned with the first frame of infrared raw image, except for the first frame of infrared raw image.

[0052] S2. Based on the L-layer feature pyramid structure, extract the multi-scale features F of each infrared raw image in the infrared raw image sequence T_align. Then, based on the multi-scale features F of each infrared raw image, align the first-layer features of each image to be aligned and the reference image in the infrared raw image sequence T_align, and construct the alignment feature sequence. , This refers to the first layer alignment feature of each image to be aligned in the infrared raw image sequence T_align, where the image to be aligned refers to each infrared raw image in the infrared raw image sequence T_align except for the first frame of infrared raw image. The specific steps include:

[0053] S21. Use the first frame of the infrared Raw image in the infrared Raw image sequence T_align as the reference image T. ref The remaining infrared raw images are used as images to be aligned with features, and an L-layer feature pyramid network (FPN) is constructed.

[0054] S22. Based on the L-layer feature pyramid structure, extract the feature F of each infrared raw image in each layer of the infrared raw image sequence T_align to obtain the multi-scale features of each infrared raw image. For example, in this embodiment, each infrared raw image is downsampled three times by 8x, 16x and 32x to obtain features at three scales.

[0055] S23, refer to image T ref Lth layer features The Lth layer features of the image to be aligned with the features of the kth frame The images are stitched together, and the stitched results are subjected to multi-layer convolution operations to obtain the k-th frame of the image to be aligned with the reference image T at the L-th layer. ref offset And there are:

[0056]

[0057] S24, Based on the Deformable Convolution Network (DCN) and the k-th frame image to be aligned with the reference image T ref offset , reference image T ref Lth layer features The Lth layer features of the image to be aligned with the features of the kth frame Perform feature alignment to obtain the L-th layer alignment features of the k-th frame image to be feature aligned. In this embodiment, the Lth layer alignment feature as follows:

[0058]

[0059] S25. Based on the following formula, obtain the relative reference image T of the k-th frame to be feature-aligned image on the (L-1)-th layer. ref offset And based on that offset Obtain the L-1 layer alignment features of the k-th frame image to be aligned. :

[0060]

[0061]

[0062] Where f(·) represents multi-layer convolution; upsample2 represents upsampling by 2;

[0063] S26. Repeat the step of obtaining the alignment features of the (L-1)th layer in step S25 until the first layer alignment features of the k-th frame image to be aligned are obtained. ;

[0064] S27. Repeat steps S23-S26 until the first layer of alignment features for each image to be aligned is obtained. Where K is the total number of images to be feature aligned in the infrared raw image sequence T_align, and 1≤K≤(N-1), k=1,2,...,K;

[0065] And construct a system that includes all first-layer alignment features. Alignment feature sequence This completes the first layer feature alignment for all infrared Raw images;

[0066] S3, Align the feature sequences Each first-layer alignment feature is aligned with the reference image T of the infrared raw image sequence T_aligned. ref The fusion process is performed to obtain the fusion feature D, which specifically includes the following steps:

[0067] S31. Calculate the alignment feature sequence according to the following formula. The k-th first-layer alignment feature Reference image T ref Similarity α k and based on the similarity α k Obtain the fusion weight w of the k-th first-layer aligned feature k ;

[0068]

[0069] Among them, T k For the k-th first-layer alignment feature The corresponding infrared raw image;

[0070] Due to differences in jitter blur and acquisition time among multiple frames, the information of moving targets in the scene varies across different frames. Therefore, to ensure consistency between the reconstructed image and the low-resolution image, the first frame is typically used as the reference frame. Features with higher similarity across multiple frames, i.e., shared aliasing features, are given higher fusion weights to avoid artifacts of moving targets. Therefore, in this embodiment, the feature fusion weight w... k Similarity to features α k Positive correlation; the higher the similarity, the higher the feature fusion weight w. k The larger;

[0071] S32. Repeat step S31 above until the fusion weight of each first-layer alignment feature is obtained.

[0072] S33. Obtain the alignment feature sequence according to the following formula. All first-layer alignment features in the reference image T ref Fusion feature D:

[0073]

[0074] Since the features of different layers in step S2 are concatenated, multi-layer convolutional operations are performed and upsampling is performed, the first layer aligned feature obtained at the end already contains the features of each layer. Therefore, in this embodiment, only the first layer features need to be fused, and multi-layer feature fusion is not required, so as to greatly reduce the amount of computation.

[0075] S4. The fused features D are processed based on the image super-resolution reconstruction network to output a high-resolution image T_HR. The specific steps include the following:

[0076] like Figure 6 As shown, both the fused feature D and the high-resolution reference image feature Y16ref are upsampled twice using pixel shuffle; wherein, the high-resolution reference image feature Y16ref is used for feature alignment or resolution matching to guide the reconstruction process;

[0077] The upsampled fusion feature D and the high-resolution reference image feature Y16ref are fused to obtain the upsampled fusion feature. For example, feature fusion can be completed by stitching the upsampled features along the channel dimension.

[0078] The image super-resolution reconstruction network is used to reconstruct upsampled fused features to output a high-resolution image T_HR. The image super-resolution reconstruction network includes structures such as convolutional layers, activation function layers, and residual blocks. It can complete the detailed reconstruction based on multi-layer convolutional computation, Unet, or attention mechanisms to obtain the high-resolution image T_HR.

[0079] Furthermore, in this embodiment, the image super-resolution reconstruction network is trained based on the following loss function L, wherein the formula for calculating the loss function L is as follows:

[0080]

[0081] Where E represents the image super-resolution reconstruction network, I HR This represents the corresponding high-resolution image in the training data. Let t represent the low-resolution infrared raw image of the t-th frame in the infrared raw image sequence, N be the number of image frames in the infrared raw image sequence, and θ represent the learnable parameters of the network.

[0082] like Figure 7-8 As shown, the high-resolution image T_HR output in this embodiment (i.e. Figure 7 (b) of the image obtained by fusing feature maps of different gray levels in the prior art (i.e.) Figure 7 In terms of part (a) of the image, it has a higher resolution and more clearly defined image details;

[0083] And, S5, obtain the target temperature based on the reconstructed high-resolution image T_HR. In this embodiment, the target temperature can be obtained based on any existing temperature measurement model. The main process includes obtaining the temperature value of each pixel in the high-resolution image T_HR, and using the matrix composed of all temperature values ​​as the temperature matrix T, thereby determining the target temperature.

[0084] Figure 7-8 All images are infrared images of a four-bar target, with the temperature difference between the background region B and the target region p set to 3°C. Figure 8 As shown in part (a), the temperatures of the background region B and the target region p obtained from the output image based on the prior art are 30.5263℃ and 30.1053℃, respectively, with a temperature difference of approximately 0.42℃. Figure 8 As shown in part (b), the temperatures of the background region B and the target region p obtained based on the high-resolution image T_HR output in this embodiment are 28.3158℃ and 31.2632℃, respectively, with a temperature difference of approximately 2.95℃, which is consistent with the set temperature difference. This indicates that the temperature measurement method in this embodiment can effectively improve the temperature measurement accuracy and is especially suitable for small targets or distant targets.

[0085] Therefore, in this embodiment, infrared raw image sequences with pixel offsets can be obtained through two image sequence construction methods. That is, by controlling the infrared detector or lens to perform precise pixel offset at the sub-pixel level within the array range, or by using human muscle tremors, multiple low-resolution infrared raw images with pixel offsets in the same scene can be obtained to simulate pixel offsets in different scenes, thereby enhancing the accuracy of infrared temperature measurement results in different scenes. Furthermore, the displacement is calculated through an image registration and alignment algorithm to align multiple low-resolution images. Finally, image reconstruction is performed based on an image super-resolution reconstruction network to fuse the multiple aligned low-resolution raw images to generate a high-resolution image T_HR with higher spatial resolution. The target temperature is then calculated based on the temperature measurement model to significantly improve the temperature measurement accuracy of targets (especially small targets and distant targets).

[0086] Meanwhile, the implementation of this embodiment is not limited by the size of the infrared detector pixels. Therefore, it will not cause problems such as the averaging of temperature information within a single pixel and inaccurate temperature measurement results for small or distant targets due to excessively large pixel size or insufficient number of pixels. It can significantly improve the temperature measurement performance of existing infrared temperature measurement equipment without replacing high-cost, large-array infrared detectors.

[0087] Example 2:

[0088] This embodiment provides an infrared temperature measurement system that can implement the infrared temperature measurement method described in Embodiment 1, such as... Figure 9 As shown, the infrared temperature measurement system includes:

[0089] Image sequence storage module 1 is used to store an infrared raw image sequence T_align containing N frames of infrared raw images, wherein the infrared raw image sequence T_align is obtained by a first image sequence construction method or a second image sequence construction method, and the process is the same as step S1.

[0090] Feature extraction module 2 extracts multi-scale features F of each infrared raw image in the infrared raw image sequence T_align based on an L-layer feature pyramid structure;

[0091] Feature alignment module 3 aligns the first-layer features of each image to be aligned and the reference image in the infrared raw image sequence T_align based on the multi-scale features F of each infrared raw image, and constructs an aligned feature sequence. The process is the same as step S2;

[0092] Feature fusion module 4, which is used to align feature sequences Each first-layer alignment feature is aligned with the reference image T of the infrared raw image sequence T_aligned. ref The process of fusing the components to obtain the fusion feature D is the same as step S3.

[0093] Image super-resolution module 5 processes the fusion feature D based on the image super-resolution reconstruction network to output a high-resolution image T_HR, and its process is the same as step S4.

[0094] And, the target temperature measurement module 6, which acquires the target temperature based on the reconstructed high-resolution image T_HR.

[0095] In summary, the present invention can obtain infrared raw image sequences with pixel offsets through two image sequence construction methods: namely, by controlling the infrared detector or lens to perform precise pixel offset at the sub-pixel level within the area array, or by utilizing human muscle tremors to obtain multiple low-resolution infrared raw images with pixel offsets in the same scene, thereby simulating pixel offset situations occurring in different scenes and enhancing the accuracy of infrared temperature measurement results in different scenes.

[0096] Further, multiple low-resolution images are aligned through image registration, and finally, image reconstruction is performed based on an image super-resolution reconstruction network to fuse the aligned low-resolution raw images to generate a high-resolution image with higher spatial resolution. The target temperature is then calculated based on a temperature measurement model to significantly improve the temperature measurement accuracy of targets (especially small targets and distant targets).

[0097] Meanwhile, the implementation of the technical solution of the present invention is not limited by the size of infrared detector pixels. Therefore, it will not cause problems such as the averaging of temperature information in a single pixel and inaccurate temperature measurement results for small or distant targets due to excessively large pixel size or insufficient number of pixels. It can significantly improve the temperature measurement performance of existing infrared temperature measurement equipment without replacing high-cost, large-array infrared detectors.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An infrared thermometry method based on pixel offset super-resolution, characterized in that, Includes the following steps: Construct an infrared raw image sequence containing N frames of infrared raw images; Based on the L-layer feature pyramid structure, multi-scale features of each infrared raw image in the infrared raw image sequence are extracted, and each image to be aligned with the first layer features of the reference image is aligned with the multi-scale features to obtain the first layer alignment features of each image to be aligned. Each first-layer alignment feature is fused with a reference image to obtain fused features; The fusion features are processed based on the image super-resolution reconstruction network to output a high-resolution image; Additionally, the target temperature is obtained based on high-resolution images.

2. The infrared temperature measurement method as described in claim 1, characterized in that, Constructing an infrared raw image sequence containing N frames of infrared raw images includes the following steps: The optical components of the infrared imaging device are moved along the X and Y directions within the area of ​​the array by a micro-displacement driving device. Each time the image is moved, one frame of infrared raw image is acquired, until N frames are acquired. Construct an infrared raw image sequence containing N frames of infrared raw images.

3. The infrared temperature measurement method as described in claim 1, characterized in that, Constructing an infrared raw image sequence containing N frames of infrared raw images includes the following steps: The infrared imaging device is driven to randomly jitter several times, and one frame of infrared raw image is acquired during each jitter, until N frames are acquired. The first frame of infrared raw image is used as the reference image T. ref And calculate the reference image T respectively. ref The accumulated row value T_rows ref Column accumulated values ​​T_cols ref and the row accumulation value T_rows of the infrared raw image to be aligned in the k-th frame. k Column accumulated values ​​T_cols k ; Based on the X-axis offset shfitX and the Y-axis offset shfitY, the infrared raw image to be aligned is translated to align with the reference image T. ref Align; After image alignment is completed, the infrared raw image sequence is constructed.

4. The infrared temperature measurement method as described in claim 3, characterized in that, Based on the X-axis offset shfitX and the Y-axis offset shfitY, the image to be aligned is translated, including the following steps: Iterate through the range of Y-axis translation values ​​to obtain the value of the current Y-axis translation mY; After each time the current Y-axis translation value mY is obtained, the row accumulation value T_rows of the infrared Raw image to be aligned in the k-th frame is incremented. k The row is shifted according to the current Y-axis shift amount mY, and the row accumulation value T_rows is calculated after each shift. k _m k ; Calculate the cumulative row count T_rows after each shift. k _m k Compared with reference image T ref The accumulated row value T_rows ref The difference between them is detaY, and the Y-direction translation mY corresponding to the minimum value of the difference detaY is determined as the Y-direction offset shfitY; Iterate through the range of values ​​for the X-direction translation to obtain the value for the current X-direction translation mX; After each time the current X-axis translation amount mX is obtained, the column accumulation value T_cols of the infrared Raw image to be aligned in the k-th frame is incremented. k The column is shifted according to the current X-axis shift amount mX, and the cumulative column value T_cols is calculated after each shift. k _m k ; Calculate the cumulative column value T_cols after each translation. k _m k Compared with reference image T ref The cumulative column values ​​T_cols ref The difference between them is detaX, and the X-direction displacement mX corresponding to the minimum value of the difference detaX is determined as the X-direction offset shfitX; Each infrared raw image to be aligned, except for the first frame, is translated by an X-axis offset shfitX and a Y-axis offset shfitY, so that each infrared raw image to be aligned is aligned with the first frame infrared raw image.

5. The infrared temperature measurement method as described in claim 1, characterized in that, Based on the multi-scale features of each infrared Raw image, the first-layer features of each image to be aligned and the reference image in the infrared Raw image sequence are aligned to obtain the first-layer alignment features of each image to be aligned. This includes the following steps: Reference image T ref Lth layer features The Lth layer features of the image to be aligned with the features of the kth frame The data is stitched together, and the stitched results are subjected to multiple convolution operations to obtain the offset of the Lth layer. And there are: ； Based on the deformable convolutional network DCN and the aforementioned offset , reference image T ref Lth layer features The Lth layer features of the image to be aligned with the features of the kth frame Perform feature alignment to obtain the L-th layer alignment features of the k-th frame image to be feature aligned. And there are: ； Based on the following formula, obtain the relative reference image T of the k-th frame to be feature-aligned image on the (L-1)-th layer. ref offset And based on that offset Obtain the L-1 layer alignment features of the k-th frame image to be aligned. : ； ； Where f(·) represents multi-layer convolution; upsample2 represents upsampling by 2; Repeat the steps for obtaining the alignment features of layer L-1 above until the first layer alignment features of the k-th frame image to be aligned are obtained. ; Repeat the above steps until the first layer of alignment features for each image to be aligned is obtained.

6. The infrared temperature measurement method as described in claim 5, characterized in that, Each first-layer alignment feature is fused with a reference image from the infrared raw image sequence to obtain fused features, including the following steps: Obtain the fusion weights for each first-layer alignment feature; All first-layer alignment features and reference image T are obtained using the following formula. ref Fusion feature D: ； Among them, w k is the fusion weight of the k-th first-layer aligned feature.

7. The infrared temperature measurement method as described in claim 6, characterized in that, The fusion weight w of the k-th first-layer alignment feature is calculated using the following formula. k : ； Among them, T k For the k-th first-layer alignment feature Corresponding infrared raw image; α k For the k-th first-layer alignment feature Reference image T ref The similarity.

8. The infrared temperature measurement method as described in claim 1, characterized in that, The image super-resolution reconstruction network processes the fused features to output a high-resolution image, including the following steps: Both the fused features and the high-resolution reference image features Y16ref were upsampled twice using pixel shuffle. The upsampled fusion features and the high-resolution reference image features Y16ref are fused together to obtain the upsampled fusion features. Furthermore, an image super-resolution reconstruction network is used to reconstruct upsampled fused features to output a high-resolution image.

9. The infrared temperature measurement method as described in claim 8, characterized in that, The image super-resolution reconstruction network is trained based on the following loss function L, where the formula for calculating the loss function L is as follows: ； Where E represents the image super-resolution reconstruction network, I HR This represents the corresponding high-resolution image in the training data. Let t represent the t-th low-resolution infrared raw image in the infrared raw image sequence, N be the number of image frames in the infrared raw image sequence, and θ represent the learnable parameters of the network.

10. An infrared temperature measurement system, characterized in that, include: The feature extraction module extracts multi-scale features F of each infrared raw image in the infrared raw image sequence based on an L-layer feature pyramid structure. The feature alignment module aligns each image to be aligned with the first layer features of the reference image based on multi-scale features F. The feature fusion module is used to fuse each first-layer aligned feature with the reference image to obtain fused features; The image super-resolution module processes the fused features based on the image super-resolution reconstruction network to output a high-resolution image; And, a target temperature measurement module, which acquires the target temperature based on the reconstructed high-resolution image.

Images