A thermal infrared image stitching method, device and equipment that retains temperature information

Abstract

The present invention discloses a thermal infrared image stitching method, device and equipment that retains temperature information, and relates to the field of image processing technology. The method includes: obtaining a thermal infrared grayscale image to be stitched and a temperature information matrix; stitching a plurality of thermal infrared grayscale images to be stitched to obtain a complete thermal infrared grayscale image after stitching; extracting a mapping relationship between the thermal infrared grayscale image and the complete thermal infrared grayscale image; performing an affine transformation on the temperature information matrix based on the mapping relationship to obtain a complete temperature information matrix. The present invention can capture the changes in pixels in the thermal infrared grayscale image during the stitching process by extracting the mapping relationship between the thermal infrared grayscale image and the complete thermal infrared grayscale image, and performing an affine transformation on the temperature information matrix based on the mapping change, so that the elements in the temperature information matrix can be synchronized with the changes in pixels in the thermal infrared grayscale image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a thermal infrared image stitching method, device and equipment that retains temperature information. Background Art

[0002] With the development of drone technology, thermal infrared remote sensing is increasingly used in agriculture, forestry and urban ecology. Due to the influence of parameters such as the field of view of the thermal infrared sensor itself, continuous operation is required to obtain thermal infrared remote sensing images over a large area. Therefore, the acquired images need to be further stitched to obtain the thermal infrared image of the area.

[0003] Currently, when thermal infrared images are stitched, they are generally stitched based on grayscale: the infrared radiation emitted by the object is captured by a thermal infrared camera and converted into grayscale. The corresponding pixels between different images are fused together according to their grayscale values ​​to form a complete grayscale image.

[0004] Although the grayscale-based stitching method can ensure the alignment of sub-images during the stitching process, the temperature information is lost because the grayscale level does not directly correspond to the temperature value. If the stitching is performed directly based on the thermal infrared remote sensing temperature in the thermal infrared image, it is difficult to achieve sub-image alignment. Summary of the invention

[0005] Based on this, in order to solve the technical problem of the conflict between retaining temperature information and aligning sub-images in the thermal infrared image stitching process in the prior art, the present invention provides a thermal infrared image stitching method, device and equipment that retains temperature information.

[0006] The present invention provides a thermal infrared image stitching method for retaining temperature information, comprising:

[0007] Acquire a plurality of thermal infrared grayscale images to be stitched, and a plurality of temperature information matrices corresponding to the plurality of thermal infrared grayscale images to be stitched; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image;

[0008] All thermal infrared grayscale images to be spliced ​​are spliced ​​to obtain a spliced ​​complete thermal infrared grayscale image; a mapping relationship between pixel points of several thermal infrared grayscale images to be spliced ​​and pixel points of the complete thermal infrared grayscale image is extracted;

[0009] Based on the mapping relationship, the temperature information matrix is ​​affine transformed to obtain a complete temperature information matrix.

[0010] Furthermore, the extracting of the mapping relationship between the pixel points of the plurality of thermal infrared grayscale images to be stitched and the pixel points of the complete thermal infrared grayscale image specifically includes:

[0011] Obtaining grid index information of a plurality of thermal infrared grayscale images to be stitched, where the grid index information is used to identify pixel points in the thermal infrared grayscale images;

[0012] Create a blank two-dimensional mask matrix of the same size as the complete thermal infrared grayscale image, where each element in the blank two-dimensional mask matrix corresponds to a pixel point in the complete thermal infrared grayscale image;

[0013] Applying an inverse affine transformation to each pixel point in the complete thermal infrared grayscale image to obtain the thermal infrared grayscale image pixel point corresponding to each pixel point in the complete thermal infrared grayscale image;

[0014] The grid index information of the thermal infrared grayscale image pixel corresponding to each pixel is stored in the corresponding element position in the blank two-dimensional mask matrix;

[0015] When all element positions in the blank two-dimensional mask matrix store grid index information of thermal infrared grayscale image pixel points, a non-blank two-dimensional mask matrix is ​​obtained, which is used to characterize the mapping relationship between the pixel points of several thermal infrared grayscale images to be spliced ​​and the pixel points of the complete thermal infrared grayscale image.

[0016] Furthermore, performing affine transformation on the temperature information matrix based on the mapping relationship specifically includes:

[0017] Constructing a blank two-dimensional temperature matrix of the same size as the non-blank two-dimensional mask matrix, based on the grid index information in the non-blank two-dimensional mask matrix, reading the thermal infrared remote sensing temperature in the temperature information matrix corresponding to the grid index information, and storing it in the blank two-dimensional temperature matrix;

[0018] When all elements in the blank two-dimensional temperature matrix store thermal infrared remote sensing temperatures, a complete temperature information matrix with a size consistent with the complete thermal infrared grayscale image is obtained.

[0019] Furthermore, the step of acquiring the temperature information matrix includes:

[0020] Acquire continuous sub-thermal infrared images in the area to be stitched at preset intervals, and measure the real temperature of objects in the area to be stitched;

[0021] Read the thermal infrared remote sensing temperature of the sub-thermal infrared image. When the thermal infrared remote sensing temperature does not match the real temperature, correct the thermal infrared remote sensing temperature to make the thermal infrared remote sensing temperature Tr of the object consistent with the real temperature, and obtain the temperature information matrix.

[0022] Furthermore, the splicing of all thermal infrared grayscale images to be spliced ​​is performed using a global similarity prior algorithm.

[0023] The present invention provides a thermal infrared image stitching device for retaining temperature information, comprising:

[0024] A data acquisition module is used to obtain a plurality of thermal infrared grayscale images to be spliced, and a plurality of temperature information matrices corresponding to the plurality of thermal infrared grayscale images to be spliced; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image;

[0025] A grayscale mapping module is used to stitch all thermal infrared grayscale images to be stitched together to obtain a complete spliced ​​thermal infrared grayscale image; extract the mapping relationship between the pixel points of several thermal infrared grayscale images to be stitched together and the pixel points of the complete thermal infrared grayscale image;

[0026] The temperature mapping module is used to perform affine transformation on the temperature information matrix based on the mapping relationship to obtain a complete temperature information matrix.

[0027] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the thermal infrared image stitching method retaining temperature information when executing the program.

[0028] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0029] In the thermal infrared image stitching method for retaining temperature information provided by the present invention, by constructing a mapping relationship between the pixels of several thermal infrared grayscale images to be stitched and the pixels of the complete thermal infrared grayscale image, the changes in the pixels in the thermal infrared grayscale image during the stitching process can be captured. Based on this mapping change, an affine transformation is performed on the temperature information matrix, so that the elements in the temperature information matrix, that is, the changes in the thermal infrared remote sensing temperature, can be synchronized with the changes in the pixels in the thermal infrared grayscale image. While retaining the temperature information, the obtained complete temperature information matrix can be kept consistent with the complete thermal infrared image after stitching to the greatest extent, thereby ensuring the alignment of the temperature information matrix to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1A detailed flow chart of a thermal infrared image stitching method that retains temperature information provided by the present invention;

[0032] Figure 2 A schematic diagram of the overall process of a thermal infrared image stitching method that retains temperature information provided by the present invention;

[0033] Figure 3 A schematic diagram of a process for obtaining a mapping relationship between pixels of a thermal infrared grayscale image and pixels of a complete thermal infrared grayscale image provided by the present invention;

[0034] Figure 4 A schematic diagram of a process for obtaining a complete temperature information matrix provided by the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Infrared image stitching is generally based on grayscale: the infrared radiation emitted by the object is captured by a thermal infrared camera, and the radiation is converted into grayscale. The corresponding pixels between different images are fused together according to their grayscale values ​​to form a complete grayscale image. The grayscale-based stitching method can ensure the alignment of sub-images during the stitching process, but the temperature information is lost because the grayscale level does not directly correspond to the temperature value. Alternatively, the stitching is directly based on the similarity of the thermal infrared remote sensing temperature in the thermal infrared image. The stitching method based on thermal infrared remote sensing temperature can retain the temperature information, but it is difficult to achieve sub-image alignment during the stitching process. The image obtained after the mainstream jigsaw algorithm or program stitches the thermal infrared image is only a grayscale image. The temperature information in the original thermal infrared image is lost, and the temperature information matrix on the region cannot be obtained.

[0037] Aiming at the problem that the temperature information will be lost after the thermal infrared images are spliced ​​by the current mainstream puzzle products and algorithms, the present invention performs the same splicing processing as the image on the original thermal infrared temperature matrix on the basis of the existing splicing algorithm to obtain the temperature information matrix of the complete area, thereby realizing the function of retaining the temperature information after the thermal infrared image splicing processing. A thermal infrared image splicing method that retains temperature information is proposed.

[0038] The technical solution provided by this embodiment is described in detail below in conjunction with the accompanying drawings.

[0039] Figure 1 and Figure 2 The flow chart of the thermal infrared image stitching method for retaining temperature information provided by this embodiment is shown. Figure 1 and Figure 2 The method is described in detail and specifically comprises the following steps:

[0040] S1: Obtain a number of thermal infrared grayscale images to be stitched, and a number of temperature information matrices corresponding to the thermal infrared grayscale images to be stitched; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image.

[0041] The unmanned aerial vehicle is equipped with a thermal infrared camera to obtain continuous original thermal infrared images x in the area at a certain interval within the set route (set the heading overlap rate and the lateral overlap rate), and the original thermal infrared image x is grayed to obtain the thermal infrared grayscale image to be spliced. At the same time, a specific object in the area (the specific object in this embodiment is a water body that is approximately a black body) is measured using a ground measurement device to measure its real temperature Tt. The original thermal infrared image x is imported into the supporting software of the thermal infrared camera, and a specific object is selected in the image frame and its temperature Tr based on thermal infrared remote sensing is read. If it does not match the actual measured temperature Tt, the original thermal infrared image temperature information is corrected in the supporting software to make the thermal infrared remote sensing temperature of the specific object consistent with the actual temperature, and then the corrected temperature information matrix is ​​exported to the local in the form of a table to obtain the temperature information matrix. Any element in the temperature information matrix corresponds to the original thermal infrared image pixel. The above operation is performed on all original thermal infrared images, that is, the original thermal infrared images to be spliced, and the obtained corrected temperature information matrix can express the real temperature information on the ground.

[0042] S2: stitching all the thermal infrared grayscale images to be stitched together to obtain a stitched complete thermal infrared grayscale image; extracting a mapping relationship between pixel points of several thermal infrared grayscale images to be stitched together and pixel points of the complete thermal infrared grayscale image.

[0043] By using the global similarity prior algorithm GSP to stitch the thermal infrared grayscale images to be stitched, a complete thermal infrared grayscale image with a complete stitched area can be obtained. There is a one-to-one correspondence between the thermal infrared grayscale image to be stitched and the temperature information matrix, that is, one pixel has one and only one temperature value corresponding to it, and the temperature value reflects the actual temperature value at the pixel. Based on the mapping relationship between the pixels of the thermal infrared grayscale image and the pixels of the complete thermal infrared grayscale image, the temperature information matrix can be affine transformed to obtain a complete temperature information matrix consistent with the complete thermal infrared grayscale image. The process of obtaining the mapping relationship between the pixels of the thermal infrared grayscale image and the pixels of the complete thermal infrared grayscale image is as follows. Figure 3As shown, the specific steps include:

[0044] S201: Obtaining grid index information of a plurality of thermal infrared grayscale images to be stitched, where the grid index information is used to identify pixel points in the thermal infrared grayscale images.

[0045] S202: Create a blank two-dimensional mask matrix of the same size as the complete thermal infrared grayscale image, where each element in the blank two-dimensional mask matrix corresponds to a pixel point in the complete thermal infrared grayscale image.

[0046] S203: Applying an inverse affine transformation to each pixel point in the complete thermal infrared grayscale image to obtain the thermal infrared grayscale image pixel point corresponding to each pixel point in the complete thermal infrared grayscale image.

[0047] S204: storing grid index information of the thermal infrared grayscale image pixel points corresponding to each pixel point into a corresponding element position in a blank two-dimensional mask matrix.

[0048] S205: When all element positions in the blank two-dimensional mask matrix store grid index information of pixel points of the thermal infrared grayscale image, a non-blank two-dimensional mask matrix is ​​obtained for representing the mapping relationship between pixel points of several thermal infrared grayscale images to be spliced ​​and pixel points of the complete thermal infrared grayscale image.

[0049] S3: Perform affine transformation on the temperature information matrix based on the mapping relationship to obtain a complete temperature information matrix. The process of obtaining a complete temperature information matrix is ​​as follows: Figure 4 As shown, the specific steps include:

[0050] S301: Construct a blank two-dimensional temperature matrix of the same size as the non-blank two-dimensional mask matrix, read the thermal infrared remote sensing temperature in the temperature information matrix corresponding to the grid index information based on the grid index information in the non-blank two-dimensional mask matrix, and store it in the blank two-dimensional temperature matrix.

[0051] S302: When all elements in the blank two-dimensional temperature matrix store thermal infrared remote sensing temperatures, a complete temperature information matrix having a size consistent with the complete thermal infrared grayscale image is obtained.

[0052] By importing the complete thermal infrared grayscale image and the complete temperature information matrix into the ENVI Classic software, it can be observed that the pixel distribution of the two is consistent, and the textures after image and temperature visualization are also basically consistent.

[0053] The thermal infrared image stitching method that retains temperature information provided in this embodiment innovatively solves the problem of temperature information loss after thermal infrared remote sensing images are stitched based on inverse affine transformation and affine change. While the thermal infrared images are stitched, a complete temperature information matrix that can reflect the real temperature information is generated, which can provide great convenience for subsequent processing of thermal infrared images and extraction of the temperature of a specified area. Because by constructing a mapping relationship between the pixels of several thermal infrared grayscale images to be stitched and the pixels of the complete thermal infrared grayscale image, the changes in the pixels in the thermal infrared grayscale image during the stitching process can be captured. Based on this mapping change, an affine transformation is performed on the temperature information matrix, which can synchronize the elements in the temperature information matrix, that is, the changes in the thermal infrared remote sensing temperature with the changes in the pixels in the thermal infrared grayscale image. While retaining the temperature information, the obtained complete temperature information matrix can be kept consistent with the complete thermal infrared image after stitching to the greatest extent, thereby ensuring the alignment of the temperature information matrix to the greatest extent.

[0054] The above is a thermal infrared image stitching method for retaining temperature information provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding thermal infrared image stitching device for retaining temperature information, including:

[0055] The data acquisition module is used to obtain a number of thermal infrared grayscale images to be stitched, and a number of temperature information matrices corresponding to the thermal infrared grayscale images to be stitched; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image.

[0056] The grayscale mapping module is used to stitch all the thermal infrared grayscale images to be stitched together to obtain a complete spliced ​​thermal infrared grayscale image; and to extract the mapping relationship between the pixel points of several thermal infrared grayscale images to be stitched together and the pixel points of the complete thermal infrared grayscale image.

[0057] The temperature mapping module is used to perform affine transformation on the temperature information matrix based on the mapping relationship to obtain a complete temperature information matrix.

[0058] The specific definition of the thermal infrared image stitching device that retains temperature information can be found in the definition of the thermal infrared image stitching method that retains temperature information above, and will not be repeated here. Each module in the above-mentioned thermal infrared image stitching device that retains temperature information can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0059] The present invention also provides a structure of a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned thermal infrared image stitching method that retains temperature information.

[0060] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0061] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A thermal infrared image stitching method that retains temperature information, characterized in that: include: Acquire a plurality of thermal infrared grayscale images to be stitched, and a plurality of temperature information matrices corresponding to the plurality of thermal infrared grayscale images to be stitched; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image; All the thermal infrared grayscale images to be stitched are stitched together to obtain a complete spliced ​​thermal infrared grayscale image; Extracting the mapping relationship between the pixel points of several thermal infrared grayscale images to be stitched and the pixel points of the complete thermal infrared grayscale image; specifically including: Obtaining grid index information of a plurality of thermal infrared grayscale images to be stitched, where the grid index information is used to identify pixel points in the thermal infrared grayscale images; Create a blank two-dimensional mask matrix of the same size as the complete thermal infrared grayscale image, where each element in the blank two-dimensional mask matrix corresponds to a pixel point in the complete thermal infrared grayscale image; Applying an inverse affine transformation to each pixel point in the complete thermal infrared grayscale image to obtain the thermal infrared grayscale image pixel point corresponding to each pixel point in the complete thermal infrared grayscale image; The grid index information of the thermal infrared grayscale image pixel corresponding to each pixel is stored in the corresponding element position in the blank two-dimensional mask matrix; When all element positions in the blank two-dimensional mask matrix store grid index information of pixel points of the thermal infrared grayscale image, a non-blank two-dimensional mask matrix is ​​obtained for representing the mapping relationship between pixel points of the plurality of thermal infrared grayscale images to be spliced ​​and pixel points of the complete thermal infrared grayscale image; Based on the mapping relationship, the temperature information matrix is ​​affine transformed to obtain a complete temperature information matrix; specifically, it includes: Constructing a blank two-dimensional temperature matrix of the same size as the non-blank two-dimensional mask matrix, based on the grid index information in the non-blank two-dimensional mask matrix, reading the thermal infrared remote sensing temperature in the temperature information matrix corresponding to the grid index information, and storing it in the blank two-dimensional temperature matrix; When all elements in the blank two-dimensional temperature matrix store thermal infrared remote sensing temperatures, a complete temperature information matrix with a size consistent with the complete thermal infrared grayscale image is obtained.

2. The thermal infrared image stitching method for retaining temperature information according to claim 1, characterized in that: The step of acquiring the temperature information matrix comprises: Acquire continuous sub-thermal infrared images in the area to be stitched at preset intervals, and measure the real temperature of objects in the area to be stitched; Read the thermal infrared remote sensing temperature of the sub-thermal infrared image. When the thermal infrared remote sensing temperature does not match the real temperature, correct the thermal infrared remote sensing temperature to make the thermal infrared remote sensing temperature Tr of the object consistent with the real temperature, and obtain the temperature information matrix.

3. The thermal infrared image stitching method for retaining temperature information according to claim 1, characterized in that: The splicing of all thermal infrared grayscale images to be spliced ​​is performed using a global similarity prior algorithm.

4. A thermal infrared image stitching device that retains temperature information, characterized in that: include: A data acquisition module is used to obtain a plurality of thermal infrared grayscale images to be spliced, and a plurality of temperature information matrices corresponding to the plurality of thermal infrared grayscale images to be spliced; the grayscale value of each pixel in the thermal infrared grayscale image is obtained by graying the infrared radiation, and each element in the temperature information matrix represents the thermal infrared remote sensing temperature of each pixel in the corresponding thermal infrared grayscale image; A grayscale mapping module is used to stitch all thermal infrared grayscale images to be stitched together to obtain a complete spliced ​​thermal infrared grayscale image; Extracting the mapping relationship between the pixel points of several thermal infrared grayscale images to be stitched and the pixel points of the complete thermal infrared grayscale image; specifically including: Obtaining grid index information of a plurality of thermal infrared grayscale images to be stitched, where the grid index information is used to identify pixel points in the thermal infrared grayscale images; Create a blank two-dimensional mask matrix of the same size as the complete thermal infrared grayscale image, where each element in the blank two-dimensional mask matrix corresponds to a pixel point in the complete thermal infrared grayscale image; Applying an inverse affine transformation to each pixel point in the complete thermal infrared grayscale image to obtain the thermal infrared grayscale image pixel point corresponding to each pixel point in the complete thermal infrared grayscale image; The grid index information of the thermal infrared grayscale image pixel corresponding to each pixel is stored in the corresponding element position in the blank two-dimensional mask matrix; When all element positions in the blank two-dimensional mask matrix store grid index information of pixel points of the thermal infrared grayscale image, a non-blank two-dimensional mask matrix is ​​obtained for representing the mapping relationship between pixel points of the plurality of thermal infrared grayscale images to be spliced ​​and pixel points of the complete thermal infrared grayscale image; The temperature mapping module is used to perform affine transformation on the temperature information matrix based on the mapping relationship to obtain a complete temperature information matrix; specifically, it includes: Constructing a blank two-dimensional temperature matrix of the same size as the non-blank two-dimensional mask matrix, based on the grid index information in the non-blank two-dimensional mask matrix, reading the thermal infrared remote sensing temperature in the temperature information matrix corresponding to the grid index information, and storing it in the blank two-dimensional temperature matrix; When all elements in the blank two-dimensional temperature matrix store thermal infrared remote sensing temperatures, a complete temperature information matrix with a size consistent with the complete thermal infrared grayscale image is obtained.

5. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 3 is implemented.

Images