A thermal imaging image stitching method, device, equipment and medium
By extracting temperature blocks using temperature distribution information in thermal imaging image stitching, the problems of difficult feature extraction and low matching accuracy in traditional methods are solved, achieving high-quality thermal imaging image stitching that is suitable for building safety inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI THERMAL IMAGING TECH CO LTD
- Filing Date
- 2026-03-31
- Publication Date
- 2026-07-03
AI Technical Summary
Existing visible light image stitching techniques cannot effectively reveal the thermal characteristics of thermal imaging images, leading to difficulties in feature extraction, decreased matching accuracy, and poor image stitching results. In particular, they cannot meet the requirements when analyzing the thermal insulation performance and humidity distribution of building walls.
By acquiring two thermal imaging images of the same target object, temperature blocks in the overlapping areas of the images are determined using temperature distribution information. Image similarity matching and fusion are then performed, and temperature blocks are extracted using temperature distribution information to improve image alignment accuracy.
It improves the accuracy of thermal imaging image stitching, ensures the quality of the stitched image, and can more accurately reflect the thermal characteristics of the target object. It is suitable for evaluating the thermal insulation performance and detecting water seepage on the exterior facade of buildings.
Smart Images
Figure CN122335535A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a thermal imaging image stitching method, apparatus, device, and medium. Background Technology
[0002] Currently, image stitching technology is mainly based on visible light images. It extracts, registers, and fuses features from multiple images based on optical characteristics such as color, edge, and texture to generate high-resolution panoramic images.
[0003] Panoramic images obtained by visible light stitching can only reflect external structural information and cannot reveal its thermal properties. For example, when analyzing the thermal insulation performance and humidity distribution of building walls, panoramic images of visible light cannot meet the needs of thermal analysis.
[0004] Thermal imaging images are generated based on the energy of infrared radiation. The optical features of thermal images, such as color, edges, and texture, are not readily apparent. For example, due to the thermal diffusion effect, the temperature boundaries of thermal images are gradual, making traditional edge features prone to failure. Using visible light image stitching methods would lead to difficulties in feature extraction, decreased matching accuracy, poor image stitching results, obvious seams, or geometric distortion. Summary of the Invention
[0005] This invention provides a method, apparatus, device, and medium for stitching thermal imaging images, which can improve the accuracy of thermal imaging image stitching and ensure the image quality of the stitched thermal imaging images.
[0006] According to one aspect of the present invention, an embodiment of the present invention provides a thermal imaging image stitching method, the method comprising: Acquire a first thermal imaging image and a second thermal imaging image for the same target object; wherein the first thermal imaging image and the second thermal imaging image have an overlapping area; In the image overlap region of the first thermal imaging image, a first temperature block of the first thermal imaging image is determined based on the temperature distribution information of the first thermal imaging image; In the image overlap region of the second thermal imaging image, a second temperature block of the second thermal imaging image is determined based on the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image. The first temperature block and the second temperature block are subjected to image similarity matching to obtain the image matching result; Based on the image matching result, the first thermal imaging image and the second thermal imaging image are fused to obtain a thermal imaging fusion result.
[0007] According to another aspect of the present invention, embodiments of the present invention also provide a thermal imaging image stitching device, the device comprising: An image acquisition module is used to acquire a first thermal imaging image and a second thermal imaging image of the same target object; wherein the first thermal imaging image and the second thermal imaging image have an image overlap region; The first extraction module is used to determine a first temperature block of the first thermal imaging image in the image overlap area of the first thermal imaging image based on the temperature distribution information of the first thermal imaging image. The second extraction module is used to determine a second temperature block of the second thermal imaging image in the image overlap region of the second thermal imaging image based on the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image. The image matching module is used to perform image similarity matching between the first temperature block and the second temperature block to obtain the image matching result; The image fusion module is used to fuse the first thermal imaging image and the second thermal imaging image according to the image matching result to obtain a thermal imaging fusion result.
[0008] According to another aspect of the present invention, embodiments of the present invention also provide a thermal imaging image stitching device, the thermal imaging image stitching device comprising: At least one processor; and A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the thermal imaging image stitching method of any embodiment of the present invention.
[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the thermal imaging image stitching method of any embodiment of the present invention.
[0010] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the thermal imaging image stitching method according to any embodiment of the present invention.
[0011] The technical solution of this invention determines the overlapping area of two images based on the shooting parameters, extracts temperature blocks in the overlapping area based on temperature distribution information, and extracts temperature blocks from the thermal imaging image using temperature distribution information. Compared with the traditional method of extracting feature regions based on image texture features and edge features, this method is easier and more accurate. Aligning the two images using temperature blocks can improve the accuracy of image alignment. Furthermore, temperature fusion is performed on the aligned images, solving the problems of inaccurate matching and poor image stitching effect in traditional stitching methods. This improves the stitching accuracy of thermal imaging images and ensures the image quality of the stitched thermal imaging image.
[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a flowchart of a thermal imaging image stitching method provided according to an embodiment of the present invention; Figure 2 This is a flowchart of a thermal imaging image stitching method provided according to an embodiment of the present invention; Figure 3 This is a structural diagram of a thermal imaging image stitching device provided according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a thermal imaging image stitching device provided in an embodiment of the present invention. Detailed Implementation
[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0017] The acquisition, storage, and application of thermal imaging images and other related technologies in the technical solutions of this invention comply with relevant laws and regulations and do not violate public order and good morals.
[0018] Figure 1 This is a flowchart illustrating a thermal imaging image stitching method provided in an embodiment of the present invention. This embodiment is applicable to panoramic stitching of thermal imaging images. The method can be executed by a thermal imaging image stitching device, which can be implemented in hardware and / or software. This thermal imaging image stitching device can be configured in a server.
[0019] See Figure 1 The thermal imaging image stitching method shown includes: S101. Acquire a first thermal imaging image and a second thermal imaging image for the same target object; wherein the first thermal imaging image and the second thermal imaging image have an image overlap area.
[0020] The target object can be any object captured by the thermal imaging image. The first thermal imaging image can be the one captured earlier. The second thermal imaging image can be the one captured later. The first thermal imaging image can be the frame preceding the second thermal imaging image. The second thermal imaging image can be the frame following the first thermal imaging image. The overlapping area can be the region in the first and second thermal imaging images where the image content is the same.
[0021] When acquiring thermal images of a target object, the thermal imaging device is controlled to continuously capture images, such that adjacent frames of thermal imaging images have an overlapping area. Optionally, the proportion of the overlapping area is not less than 30% of the thermal imaging image.
[0022] For example, the thermal imaging device is mounted on a tripod or pan-tilt head at a certain distance from the target object. The thermal imaging device is controlled to capture one frame of thermal imaging image at intervals of approximately 0.3 meters per step along the horizontal direction of the target object. It is ensured that the overlapping area of two adjacent thermal imaging images is not less than 30%, and the sequence number and shooting time of each thermal imaging image are recorded.
[0023] S102. In the image overlap area of the first thermal imaging image, a first temperature block of the first thermal imaging image is determined according to the temperature distribution information of the first thermal imaging image.
[0024] The temperature distribution information can be information describing the characteristics of the temperature distribution. For example, isotherm distribution can be used as temperature distribution information. The first temperature block can be a region block with temperature characteristics in the first thermal imaging image.
[0025] A thermal imaging image is an image that describes the temperature of an object. The temperature distribution of a target object can be obtained through thermal imaging images.
[0026] In practice, the contours of characteristic structures within a target object can often be identified using thermal imaging images. This is because regions of different materials or surface emissivity within the target object exhibit different temperatures, which are reflected in the thermal imaging image. By analyzing the temperature distribution of a first thermal imaging image, the image region forming the characteristic structure within the first thermal imaging image is extracted and used as a first temperature block. For example, when the target object is the exterior facade of a building, the temperature of windows differs from that of the surrounding walls, and the contours of windows can be identified in the resulting thermal imaging image. The image region of the window in the first thermal imaging image can be used as the first temperature block.
[0027] S103. In the image overlap region of the second thermal imaging image, a second temperature block of the second thermal imaging image is determined according to the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image.
[0028] The second temperature block can be a region with temperature characteristics in the second thermal imaging image. The imaging parameters can be the parameters used to capture the thermal imaging image.
[0029] The image overlap area is determined by the settings during shooting. For example, based on the thermal imaging equipment's settings, if the actual width of the target object observed in the thermal imaging equipment is 6 meters, and one image is taken every 3 meters horizontally during shooting, the image overlap area of the first thermal image can be determined as the right half of the image. The image overlap area of the second thermal image can be determined as the left half of the image.
[0030] S104. Perform image similarity matching on the first temperature block and the second temperature block to obtain the image matching result.
[0031] The image matching result can be the result of matching the first temperature block and the second temperature block.
[0032] In some embodiments, if there is only one first temperature block and one second temperature block, the image matching result is either a successful match or a failed match.
[0033] In some embodiments, if there are multiple first temperature blocks and multiple second temperature blocks, the image matching result is a temperature block matching pair formed by the first temperature block and the matched second temperature block.
[0034] Optionally, Euclidean distance is used as a similarity metric. The Euclidean distance between the feature vectors of the first temperature block and the second temperature block is calculated, and this distance is used to determine whether the first and second temperature blocks match. When multiple second temperature blocks exist, the Euclidean distance between the first temperature block and each of the second temperature blocks is calculated to obtain the optimal and second-optimal Euclidean distances. When the optimal Euclidean distance is less than the second-optimal Euclidean distance by a distance ratio threshold, the second temperature block corresponding to the optimal Euclidean distance is considered to have matched the first temperature block. The distance ratio threshold can be 0.8.
[0035] S105. Based on the image matching result, the first thermal imaging image and the second thermal imaging image are fused to obtain a thermal imaging fusion result.
[0036] The thermal imaging fusion result can be a fusion of the first thermal imaging image and the second thermal imaging image into a single image. In practice, there are usually more than two thermal imaging images of the target object. However, all of them can be fused using the thermal imaging image stitching method of this embodiment. For example: first, select thermal imaging image number 1 as the first thermal imaging image, select thermal imaging image number 2 as the second thermal imaging image, and fuse them to obtain the thermal imaging fusion result. Then, use the thermal imaging fusion result as the first thermal imaging image, select thermal imaging image number 3 as the second thermal imaging image, and fuse them using this method. This process continues until the last thermal imaging image is fused, resulting in a panoramic thermal imaging image of the target object.
[0037] When there is a first temperature block with a matching second temperature block, that is, when there is at least one temperature block matching pair, the first thermal imaging image and the second thermal imaging image can be fused.
[0038] One feasible method is as follows: A coordinate transformation matrix is obtained based on the coordinates of the first temperature block in the first thermal imaging image and the coordinates of the second temperature block in the second thermal imaging image. The second thermal imaging image is then transformed according to the coordinate transformation matrix, so that the second thermal imaging image and the first thermal imaging image are in the same coordinate system. The first thermal imaging image and the transformed second thermal imaging image are then spatially aligned based on the first and second temperature blocks, so that the first and second temperature blocks coincide in space. The overlapping areas of the spatially aligned first and second thermal imaging images are then fused to obtain the thermal imaging fusion result.
[0039] In an optional embodiment, fusing the first thermal imaging image and the second thermal imaging image according to the feature matching result to obtain a thermal imaging fusion result includes: determining a first fusion region in the first thermal imaging image and a second fusion region in the second thermal imaging image based on a first temperature block in the feature matching result and a second temperature block matching the first temperature block; performing weighted fusion of the temperatures corresponding to each pixel in the first fusion region and the temperatures of each pixel in the second fusion region to obtain a temperature fusion result; converting the temperatures of each pixel in the temperature fusion result into pixel values to obtain an image fusion result; and determining the thermal imaging fusion result based on the region outside the first fusion region in the first thermal imaging image, the region outside the second fusion region in the second thermal imaging image, and the image fusion result.
[0040] The first fusion region can be the region in the first thermal imaging image that needs to be fused. The second fusion region can be the region in the second thermal imaging image that needs to be fused. The temperature fusion result can be the result obtained by fusing temperatures. The image fusion result can be the result obtained after fusing the fusion regions.
[0041] After spatially aligning the first thermal image and the second thermal image based on the first temperature block and its matching second temperature block, a first fusion region in the first thermal image and a second fusion region in the second thermal image are obtained. The temperatures corresponding to each pixel in the first fusion region and the temperatures corresponding to each pixel in the second fusion region are acquired. For each pixel, the temperatures of corresponding pixels in the first and second fusion regions are weighted and fused to obtain the fusion temperature of that pixel, thus obtaining the final temperature fusion result. Optionally, the weights for temperature fusion are determined based on the distance from the pixel to the image boundary.
[0042] Thermal imaging equipment acquires the thermal infrared energy radiated by a target object, converts it into pixel values according to preset camera parameters, and displays it on the image. Therefore, the pixel values of a thermal image represent the energy radiated by the object, i.e., the energy value. Since energy value and temperature follow the Stefan-Boltzmann law, temperature can be calculated from the energy value. Similarly, after obtaining the fusion temperature of the pixels, the corresponding energy value can be calculated. Then, according to preset camera parameters, the energy value is converted back into pixel values. Finally, the fusion temperature is reflected in the thermal imaging image, resulting in the image fusion result.
[0043] In the first thermal imaging image and the second thermal imaging image, the pixel values of other regions remain unchanged except for the first fusion region and the second fusion region.
[0044] It is evident that by fusing the temperatures of the fusion regions of the first and second thermal imaging images, the physical correctness and numerical rationality of the image fusion result are ensured. Because pixel values and true temperature have a non-linear relationship, directly weighting and fusing pixel values may lead to unreasonable temperature reconstructions and temperature distortion. However, fusing the temperatures corresponding to pixels in the first and second fusion regions ensures that the image fusion result has reliable physical meaning.
[0045] In an optional embodiment, the target object includes: the building facade; after fusing the first thermal imaging image and the second thermal imaging image according to the feature matching result, the method further includes: performing a security inspection on the building based on the fused thermal imaging image.
[0046] The target objects include building facades. When a building needs safety inspection, relying solely on visible light image stitching is insufficient for assessing performance such as thermal insulation and detecting water seepage. By stitching together thermal imaging images to obtain a panoramic temperature distribution of the building facade, and then using this panoramic temperature distribution to assess the building's thermal insulation and detect water seepage, potential risks can be identified more accurately, allowing for timely repairs.
[0047] It is evident that obtaining a panoramic temperature distribution image of a building's exterior facade can provide a reliable temperature basis for subsequent building safety inspections, thereby improving the accuracy of safety hazard identification.
[0048] The technical solution of this invention determines the overlapping area of two images based on the shooting parameters, extracts temperature blocks in the overlapping area based on temperature distribution information, and extracts temperature blocks from the thermal imaging image using temperature distribution information. Compared with the traditional method of extracting feature regions based on image texture features and edge features, this method is easier and more accurate. Aligning the two images using temperature blocks can improve the accuracy of image alignment. Furthermore, temperature fusion is performed on the aligned images, solving the problems of inaccurate matching and poor image stitching effect in traditional stitching methods. This improves the stitching accuracy of thermal imaging images and ensures the image quality of the stitched thermal imaging image.
[0049] Figure 2 This is a flowchart of a thermal imaging image stitching method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment determines a first temperature block of the first thermal imaging image in the image overlap region according to the temperature distribution information of the first thermal imaging image. The method is defined as follows: obtaining the temperature of each pixel in the image overlap region of the first thermal imaging image, calculating the temperature gradient of each pixel; when the gradient value of the temperature gradient of a pixel is greater than a gradient value threshold, the pixel is determined as a temperature feature point; and based on the temperature, position distribution, and direction of the temperature gradient of multiple temperature feature points, the temperature feature points are aggregated to obtain at least one first temperature block.
[0050] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments.
[0051] See Figure 2 The thermal imaging image stitching method shown includes: S201. Acquire a first thermal imaging image and a second thermal imaging image for the same target object; wherein the first thermal imaging image and the second thermal imaging image have an image overlap area.
[0052] S202. Obtain the temperature of each pixel in the overlapping region of the first thermal imaging image, and calculate the temperature gradient of each pixel.
[0053] Temperature gradient is a physical quantity that describes the direction and degree of temperature change in space.
[0054] Obtain the temperature of each pixel in the overlapping region, denoted as T(x,y), where x and y are the coordinates of the pixels.
[0055] For each pixel, calculate the pixel's temperature gradient.
[0056] For example, a method for calculating a temperature gradient is as follows: Calculate the horizontal temperature difference: G x= T(x+1,y)- T(x,y); Calculate the vertical temperature difference: G y = T(x,y+1)- T(x,y); The magnitude of the temperature gradient is: M 2 = G x 2 +G y 2 ; The direction of the temperature gradient is: θ(x,y) = tan -1 (G) x / G y ) In an optional embodiment, obtaining the temperature of each pixel in the overlapping region of the first thermal imaging image includes: obtaining the pixel value of each pixel in the overlapping region of the first thermal imaging image, wherein the pixel value is proportional to the thermal infrared radiation energy; and converting the pixel value of the pixel into temperature according to the thermal imaging acquisition parameters.
[0057] This process involves extracting the pixel values of each pixel in the overlapping region of the image. These pixel values are typically digitally quantized values, i.e., gray levels after analog-to-digital conversion. These pixel values exhibit a linear or approximately linear relationship with the radiant energy received by the thermal infrared sensor in the thermal imaging device.
[0058] Converting pixel values to physical temperature requires radiation value conversion based on the calibration parameters of the thermal imaging system: using the gain and bias coefficients during acquisition, pixel values are converted into radiation energy values, and the temperature corresponding to the pixel is calculated using the inverse function of Planck's radiation law or a lookup table of the relationship between radiation energy values and temperature.
[0059] As can be seen, by converting the pixel values of thermal imaging images into physical temperatures, thermal imaging images can be upgraded from relative grayscale to a true temperature field, enabling accurate calculation of key thermodynamic parameters such as temperature gradients or heat flux density.
[0060] In an optional embodiment, before obtaining the temperature of each pixel in the image overlap region of the first thermal imaging image and calculating the temperature gradient of each pixel, the method further includes: performing isolated point detection on the first thermal imaging image to obtain isolated pixels that have a temperature difference with all neighboring pixels; and replacing the temperature corresponding to the isolated pixel with the temperature corresponding to the neighboring pixels of the isolated pixel.
[0061] An isolated pixel can be a pixel that exhibits a temperature jump compared to its surrounding temperature.
[0062] Generally, in thermal imaging images, the temperature of each pixel is at least similar to the temperature of one side of the image. For example, a pixel representing the change in building material at the edge of a window may have a large temperature difference from the pixel representing the wall, but not a large temperature difference from the pixel representing the window frame.
[0063] Understandably, if the temperature of a pixel differs significantly from the temperatures of all surrounding pixels, that pixel may be a noise point. Thermal imaging devices are affected by random thermal noise, pixel drift, and quantization errors during acquisition, causing localized temperature fluctuations in the thermal image, manifesting as isolated pixels with abnormal temperatures.
[0064] To avoid affecting the accuracy of temperature block determination, isolated pixels can be replaced with their temperatures first. The temperature of the isolated pixel is then replaced based on the temperatures of its adjacent pixels. Optionally, the temperature of the isolated pixel can be replaced with the temperature of any adjacent pixel.
[0065] It is evident that by replacing the temperature of isolated pixels, the impact of noise on temperature block determination can be reduced, thereby increasing the accuracy of temperature block determination.
[0066] S203. When the gradient value of the temperature gradient of the pixel is greater than the gradient value threshold, the pixel is determined as a temperature feature point.
[0067] Here, the gradient value can be a value describing the magnitude of the temperature gradient. The gradient value threshold can be a preset value used to determine the magnitude of the gradient value. The temperature feature point can be a pixel that exhibits temperature characteristics.
[0068] When the temperature gradient value of a pixel is greater than the gradient value threshold, it indicates that the pixel experiences a significant temperature change, suggesting that the pixel is located at a point of temperature change, corresponding to a structural boundary within the target object. For example, a pixel at the boundary between a window and a wall should have a temperature gradient value greater than the gradient value threshold.
[0069] S204. Based on the temperature, location distribution and temperature gradient direction of the multiple temperature feature points, aggregate the temperature feature points to obtain at least one first temperature block.
[0070] Multiple temperature feature points can be identified in the overlapping areas of the images.
[0071] When aggregating multiple temperature feature points to form a first temperature feature block, the temperature feature points to be aggregated must meet the following conditions: ① The temperature values are similar, and the temperature difference between the temperature feature points does not exceed a set temperature threshold; ② The spatial continuity is such that the temperature feature points can be connected through adjacency, forming a connected region; ③ The directions of the temperature gradients are not contradictory, and the temperature gradient directions of each temperature feature point should both point to a certain region or both move away from a certain region. Only temperature feature points that simultaneously meet the above conditions are suitable for aggregation into a first temperature block.
[0072] In an optional embodiment, the step of aggregating the temperature feature points according to their temperature, location distribution, and the direction of the temperature gradient to obtain at least one first temperature block includes: connecting two temperature feature points if the temperature difference between them is less than a temperature difference threshold, the distance between them is less than a distance threshold, and the angle between the directions of the temperature gradients is less than an angle threshold; and determining a first temperature block as a set of multiple connected temperature feature points.
[0073] The temperature difference threshold can be a preset upper limit for the temperature difference. The distance threshold can be a preset upper limit for the distance difference. The distance threshold can be determined based on the resolution of the thermal imaging image. The angle threshold can be a preset upper limit for the angle difference in the temperature gradient direction.
[0074] First, select a temperature feature point as the target point. Then, determine whether other temperature feature points meet the following conditions: their temperature difference from the target point is less than a temperature difference threshold, their distance from the target point is less than a distance threshold, and the angle between their temperature gradient direction and the target point is less than an angle threshold. Temperature feature points that meet these conditions are identified as connected points to the target point, and the target point is connected to these connected points. Next, these connected points are identified as new target points, and conditional checks are performed on them with the remaining temperature feature points to identify new connected points. This process is repeated to determine a first temperature block.
[0075] As can be seen, by connecting two temperature feature points that meet the conditions in terms of temperature, location, and temperature gradient direction, and so on, the first temperature block is obtained by aggregation. The clear judgment conditions enable the system to more clearly complete the determination of the first temperature block, thereby improving the efficiency and accuracy of the determination of the first temperature block.
[0076] S205. In the image overlap region of the second thermal imaging image, a second temperature block of the second thermal imaging image is determined according to the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image.
[0077] The method for determining the second temperature block is the same as that for the first temperature block.
[0078] S206. Perform image similarity matching on the first temperature block and the second temperature block to obtain the image matching result.
[0079] S207. Based on the image matching result, the first thermal imaging image and the second thermal imaging image are fused to obtain a thermal imaging fusion result.
[0080] The technical solution of this invention, which extracts temperature feature points based on temperature gradients, can adapt to the temperature distribution range under different scenarios and has good scenario generalization ability. By using aggregation conditions, temperature feature points are filtered and aggregated, and discrete feature points are transformed into a first temperature block. The judgment conditions are clear and highly executable, which improves the determination efficiency of the first temperature block. The aggregation logic is clear and reduces the complexity of temperature feature extraction.
[0081] Figure 3 This is a schematic diagram of a thermal imaging image stitching device provided in an embodiment of the present invention. The present invention is applicable to panoramic stitching of thermal imaging images. This device can execute a thermal imaging image stitching method and can be implemented in hardware and / or software.
[0082] See Figure 3 The thermal imaging image stitching device shown includes: The image acquisition module 301 is used to acquire a first thermal imaging image and a second thermal imaging image for the same target object; wherein the first thermal imaging image and the second thermal imaging image have an image overlap area; The first extraction module 302 is used to determine a first temperature block of the first thermal imaging image in the image overlap area of the first thermal imaging image based on the temperature distribution information of the first thermal imaging image. The second extraction module 303 is used to determine a second temperature block of the second thermal imaging image in the image overlap region of the second thermal imaging image based on the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image. Image matching module 304 is used to perform image similarity matching between the first temperature block and the second temperature block to obtain image matching results; The image fusion module 305 is used to fuse the first thermal imaging image and the second thermal imaging image according to the image matching result to obtain a thermal imaging fusion result.
[0083] The technical solution of this invention determines the overlapping area of two images based on the shooting parameters, extracts temperature blocks in the overlapping area based on temperature distribution information, and extracts temperature blocks from the thermal imaging image using temperature distribution information. Compared with the traditional method of extracting feature regions based on image texture features and edge features, this method is easier and more accurate. Aligning the two images using temperature blocks can improve the accuracy of image alignment. Furthermore, temperature fusion is performed on the aligned images, solving the problems of inaccurate matching and poor image stitching effect in traditional stitching methods. This improves the stitching accuracy of thermal imaging images and ensures the image quality of the stitched thermal imaging image.
[0084] In an optional embodiment, the first extraction module 302 includes: The gradient calculation unit is used to obtain the temperature of each pixel in the image overlap region of the first thermal imaging image and calculate the temperature gradient of each pixel. A point determination unit is used to determine the pixel as a temperature feature point when the gradient value of the temperature gradient of the pixel is greater than the gradient value threshold. The block determination unit is used to aggregate the temperature feature points according to the temperature, position distribution and temperature gradient direction of the multiple temperature feature points to obtain at least one first temperature block.
[0085] In an optional embodiment, the block determination unit includes: The point-connected sub-unit is used to connect two temperature feature points if the temperature difference is less than the temperature difference threshold, the distance is less than the distance threshold, and the angle between the directions of the temperature gradient is less than the angle threshold. The block determination subunit is used to determine a first temperature block by identifying multiple temperature feature points that are connected.
[0086] In an optional embodiment, the gradient calculation unit includes: A pixel acquisition subunit is used to acquire the pixel value of each pixel in the image overlap region of the first thermal imaging image, wherein the pixel value is proportional to the thermal infrared radiation energy. The temperature calculation subunit is used to convert the pixel value of the pixel point into temperature based on the thermal imaging acquisition parameters.
[0087] In an optional embodiment, it further includes: An isolated point determination unit is used to detect isolated points in the first thermal imaging image and obtain isolated pixels that have temperature differences from all adjacent pixels. A temperature replacement unit is used to replace the temperature corresponding to the isolated pixel with the temperature corresponding to the adjacent pixels of the isolated pixel.
[0088] Optionally, the image fusion module 305 includes: The fusion region determination unit is used to determine a first fusion region in the first thermal imaging image and a second fusion region in the second thermal imaging image based on a first temperature block in the feature matching result and a second temperature block that matches the first temperature block. The temperature fusion unit is used to perform weighted fusion of the temperature corresponding to each pixel in the first fusion region and the temperature of each pixel in the second fusion region to obtain the temperature fusion result. A pixel conversion unit is used to convert the temperature of each pixel in the temperature fusion result into a pixel value to obtain the image fusion result. The image determination unit is configured to determine a thermal imaging fusion result based on the region outside the first fusion region in the first thermal imaging image, the region outside the second fusion region in the second thermal imaging image, and the image fusion result.
[0089] Optionally, the target object includes: the building facade; after fusing the first thermal imaging image and the second thermal imaging image according to the feature matching result, the method further includes: performing a security inspection on the building based on the fused thermal imaging image.
[0090] The thermal imaging image stitching device provided in this embodiment of the invention can execute the thermal imaging image stitching method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the thermal imaging image stitching method.
[0091] Figure 4 A schematic diagram of the structure of a thermal imaging image stitching device 400 that can be used to implement an embodiment of the present invention is shown.
[0092] like Figure 4 As shown, the thermal imaging image stitching device 400 includes at least one processor 401 and a memory, such as a read-only memory 402 or a random access memory 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 402 or loaded from the storage unit 408 into the random access memory 403. The random access memory 403 can also store various programs and data required for the operation of the thermal imaging image stitching device 400. The processor 401, read-only memory 402, and random access memory 403 are interconnected via a bus 404. An input / output interface 405 is also connected to the bus 404.
[0093] Multiple components in the thermal imaging image stitching device 400 are connected to the input / output interface 405, including: an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless transceiver, etc. The communication unit 409 allows the thermal imaging image stitching device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0094] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as thermal imaging image stitching methods.
[0095] In some embodiments, the thermal imaging image stitching method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the thermal imaging image stitching device 400 via read-only memory 402 and / or communication unit 409. When the computer program is loaded into random access memory 403 and executed by processor 401, one or more steps of the thermal imaging image stitching method described above may be performed. Alternatively, in other embodiments, processor 401 may be configured to perform the thermal imaging image stitching method by any other suitable means (e.g., by means of firmware).
[0096] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0097] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0098] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, flash memory, optical fiber, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0099] To provide user interaction, the systems and techniques described herein can be implemented on an operational detection device. This thermal imaging image stitching device includes: a display device (e.g., a cathode ray tube or liquid crystal monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the thermal imaging image stitching device. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0100] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0101] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.
[0102] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0103] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A thermal imaging image stitching method, characterized by, The method includes: Acquire a first thermal imaging image and a second thermal imaging image for the same target object; wherein the first thermal imaging image and the second thermal imaging image have an overlapping area; In the image overlap region of the first thermal imaging image, a first temperature block of the first thermal imaging image is determined based on the temperature distribution information of the first thermal imaging image; In the image overlap region of the second thermal imaging image, a second temperature block of the second thermal imaging image is determined based on the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image. The first temperature block and the second temperature block are subjected to image similarity matching to obtain the image matching result; Based on the image matching result, the first thermal imaging image and the second thermal imaging image are fused to obtain a thermal imaging fusion result.
2. The method of claim 1, wherein, Determining a first temperature block in the overlapping region of the first thermal imaging image based on the temperature distribution information of the first thermal imaging image includes: The temperature of each pixel in the overlapping region of the first thermal imaging image is obtained, and the temperature gradient of each pixel is calculated. When the gradient value of the temperature gradient of the pixel is greater than the gradient value threshold, the pixel is determined as a temperature feature point. Based on the temperature, location distribution, and temperature gradient direction of multiple temperature feature points, the temperature feature points are aggregated to obtain at least one first temperature block.
3. The method of claim 2, wherein, The step of aggregating the temperature feature points based on their temperature, location distribution, and temperature gradient direction to obtain at least one first temperature block includes: If two temperature feature points satisfy the following conditions: temperature difference is less than the temperature difference threshold, distance is less than the distance threshold, and the angle between the directions of the temperature gradients is less than the angle threshold, then the two temperature feature points are connected. Multiple temperature feature points that are connected are identified as a first temperature block.
4. The method of claim 2, wherein, The step of obtaining the temperature of each pixel in the overlapping region of the first thermal imaging image includes: Obtain the pixel value of each pixel in the overlapping region of the first thermal imaging image, wherein the pixel value is proportional to the thermal infrared radiation energy; Based on the thermal imaging acquisition parameters, the pixel values of the pixels are converted into temperature.
5. The method of claim 1, wherein, The step of fusing the first thermal imaging image and the second thermal imaging image according to the feature matching result to obtain a thermal imaging fusion result includes: Based on the first temperature block in the feature matching result and the second temperature block that matches the first temperature block, the first fusion region in the first thermal imaging image and the second fusion region in the second thermal imaging image are determined. The temperature of each pixel in the first fusion region and the temperature of each pixel in the second fusion region are weighted and fused to obtain the temperature fusion result. The temperature of each pixel in the temperature fusion result is converted into a pixel value to obtain the image fusion result; The thermal imaging fusion result is determined based on the region outside the first fusion region in the first thermal imaging image, the region outside the second fusion region in the second thermal imaging image, and the image fusion result.
6. The method of claim 2, wherein, Before acquiring the temperature of each pixel in the overlapping region of the first thermal imaging image and calculating the temperature gradient of each pixel, the method further includes: Isolated point detection is performed on the first thermal imaging image to obtain isolated pixels that have temperature differences from all adjacent pixels; Replace the temperature corresponding to the isolated pixel with the temperature corresponding to the neighboring pixels of the isolated pixel.
7. The method of claim 1, wherein, The target object includes: the exterior facade of a building; After fusing the first thermal imaging image and the second thermal imaging image based on the feature matching result, the method further includes: Safety inspections of buildings are conducted based on the fused thermal imaging images.
8. A thermal imaging image stitching apparatus, characterized by, The device includes: An image acquisition module is used to acquire a first thermal imaging image and a second thermal imaging image of the same target object; wherein the first thermal imaging image and the second thermal imaging image have an image overlap region; The first extraction module is used to determine a first temperature block of the first thermal imaging image in the image overlap area of the first thermal imaging image based on the temperature distribution information of the first thermal imaging image. The second extraction module is used to determine a second temperature block of the second thermal imaging image in the image overlap region of the second thermal imaging image based on the temperature distribution information of the second thermal imaging image; the image overlap regions of the first thermal imaging image and the second thermal imaging image are determined by the shooting parameters of the first thermal imaging image and the second thermal imaging image. The image matching module is used to perform image similarity matching between the first temperature block and the second temperature block to obtain the image matching result; The image fusion module is used to fuse the first thermal imaging image and the second thermal imaging image according to the image matching result to obtain a thermal imaging fusion result.
9. A thermal imaging image stitching apparatus, characterized by, The thermal imaging image stitching device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the thermal imaging image stitching method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the thermal imaging image stitching method according to any one of claims 1-7.