A method for mosaicking infrared images of vehicles with non-uniform speed based on temperature difference features

Abstract

The present invention belongs to the field of image stitching technology, and specifically relates to a method for stitching non-uniform vehicle infrared images based on temperature difference features. When a vehicle passes through a security inspection channel, a vehicle infrared image is captured in a single-frame manner; an ROI region is set on each frame of the vehicle infrared image; based on the ROI region of each frame of the vehicle infrared image, a column feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference, and similar columns in adjacent frames of vehicle infrared images are matched based on the column feature matrix, which are the columns to be stitched; based on the ROI region of each frame of the vehicle infrared image, a row feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference, and similar rows in adjacent frames of vehicle infrared images are matched based on the row feature matrix, which are the rows to be stitched; based on the columns and rows to be stitched, two adjacent frames of vehicle infrared images are stitched; and the stitching seam between the two adjacent frames of vehicle infrared images is smoothed to obtain a final stitched image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image stitching, and in particular relates to a method for stitching infrared images of non-uniform-speed vehicles based on temperature difference features. Background Art

[0002] In vehicle security inspection scenarios, collecting temperature information from multiple perspectives (left, right, and top) of vehicles passing through security inspection channels is a key technical means to ensure public safety and prevent potential safety hazards such as the transportation of dangerous goods. However, existing technologies based on this technology have the following problems:

[0003] For one thing, the length of the vehicle being measured (such as buses and trucks) often exceeds the imaging range of a single infrared image. Due to the resolution and lens angle limitations of medium-wave infrared array cameras, a single frame can only capture a localized area of ​​the vehicle, such as the front, rear, or a portion of the side, making it impossible to directly obtain temperature information for the entire vehicle. Relying on simple frame stacking or splicing can result in missing or redundant data due to factors such as vehicle obstruction and changes in viewing angle, compromising the integrity of the temperature field distribution.

[0004] On the other hand, vehicles typically travel at non-uniform speeds in security inspection lanes, and their speed fluctuations can cause significant positional offsets, rotations, or scaling differences between multiple frames. This dynamic change further complicates the reconstruction of the full-body temperature field: infrared images are characterized by blurred edges. Traditional static scene-based stitching algorithms, such as feature-point-based SIFT and ORB, struggle to effectively handle dynamic and blurred targets, and are prone to stitching discontinuities, ghosting, or geometric distortion. Global optimization-based stitching methods, such as optical flow and variational models, can improve accuracy but are computationally complex and struggle to meet the real-time requirements of a 25Hz frame rate.

[0005] Furthermore, infrared images inherently suffer from low contrast and limited texture information, making it difficult to match features on vehicle surface areas (such as smooth metal and painted surfaces), further limiting the applicability of existing stitching techniques. Therefore, overcoming the limitations of single-frame imaging, overcoming the interference of non-uniform motion, and achieving precise matching and stitching of vehicle surface features have become pressing technical challenges in the field of infrared vehicle security inspections. Summary of the Invention

[0006] In response to the problems in the existing technology such as insufficient single-frame imaging range, large non-uniform motion interference, and difficulty in infrared image feature matching, the present invention provides a non-uniform vehicle infrared image stitching method based on temperature difference features to achieve reconstruction of the vehicle's overall temperature field.

[0007] The technical solution of the present invention to solve the above technical problems is as follows:

[0008] The present invention provides a method for stitching infrared images of non-uniform speed vehicles based on temperature difference characteristics, comprising the following steps:

[0009] When a vehicle passes through the security inspection channel, the vehicle infrared image is collected in a single frame mode; the ROI area is set on each frame of the vehicle infrared image;

[0010] Based on the ROI area of ​​each frame of vehicle infrared image, the column feature matrix of the ROI area of ​​each frame of vehicle infrared image is calculated by normalizing the temperature difference, and similar columns in adjacent vehicle infrared images are matched based on the column feature matrix, which are the columns to be spliced;

[0011] Based on the ROI area of ​​each frame of vehicle infrared image, the row feature matrix of the ROI area of ​​each frame of vehicle infrared image is calculated by normalizing the temperature difference, and similar rows in adjacent vehicle infrared images are matched based on the row feature matrix, which are the rows to be spliced;

[0012] Based on the columns to be spliced ​​and the rows to be spliced, two adjacent frames of vehicle infrared images are spliced; and the splicing seams of the two adjacent frames of vehicle infrared images are smoothed to obtain a final spliced ​​image.

[0013] Furthermore, for each frame of vehicle infrared image, a ROI area is delineated within a predefined fixed coordinate range.

[0014] Furthermore, the column feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference based on the ROI region of each frame of the vehicle infrared image, and similar columns in adjacent vehicle infrared images are matched based on the column feature matrix, that is, the columns to be spliced, including:

[0015] Calculate the column average temperature of the ROI area of ​​each frame of the vehicle infrared image as the column reference temperature; based on the column reference temperature, calculate the normalized temperature difference for each column of the ROI area of ​​each frame of the vehicle infrared image to obtain the column feature matrix; calculate the similarity of the column feature matrices of the ROI areas of two adjacent vehicle infrared image frames, and find the column with the highest similarity in the two adjacent vehicle infrared image frames, which is the column to be spliced.

[0016] Furthermore, the similarity of the column feature matrices of the ROI regions of two adjacent vehicle infrared image frames is calculated using the mutual correlation coefficient or the Euclidean distance.

[0017] Furthermore, the row feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference, and similar rows in adjacent vehicle infrared images are matched based on the row feature matrix, that is, the rows to be spliced, including:

[0018] Calculate the average row temperature of the ROI area of ​​each frame of the vehicle infrared image as the row reference temperature. Based on the row reference temperature, calculate the normalized temperature difference for each row of the ROI area of ​​each frame of the vehicle infrared image to obtain the row feature matrix. Use the cross-correlation coefficient or Euclidean distance to calculate the similarity of the row feature matrices of the ROI areas of two adjacent vehicle infrared image frames. Find the row with the highest similarity in the two adjacent vehicle infrared image frames, which is the row to be spliced.

[0019] Furthermore, the similarity of the row feature matrices of the ROI regions of two adjacent vehicle infrared image frames is calculated using the mutual correlation coefficient or the Euclidean distance.

[0020] Furthermore, the stitching seams between two adjacent frames of vehicle infrared images are smoothed to obtain the final stitching image, including:

[0021] The specific position of the seam is determined according to the columns and rows to be spliced; a preset number of columns are selected on both sides of the seam as the area to be smoothed; and the area to be smoothed is smoothed to obtain the final spliced ​​image.

[0022] Compared with the prior art, the present invention has the following technical effects:

[0023] This invention provides a method for stitching infrared images of vehicles moving at non-uniform speeds based on temperature difference features. By extracting the temperature difference features of infrared images, it achieves precise matching and stitching of adjacent frames, eliminating the interference of non-uniform motion and producing a stitched image with a natural transition. This method offers advantages such as a wide imaging range, high matching accuracy, and natural stitching effects, making it suitable for applications in fields such as vehicle security inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 It is a schematic diagram of the process of the present invention;

[0026] Figure 2 Schematic diagram of the ROI region of a single-frame image of the present invention;

[0027] Figure 3 is the first single-frame image to be spliced ​​in the present invention;

[0028] Figure 4 is the second single-frame image to be spliced ​​in the present invention;

[0029] Figure 5 is a portion of the third single-frame image to be spliced ​​in the present invention;

[0030] Figure 6 is a portion of the fourth single-frame image to be spliced ​​in the present invention;

[0031] Figure 7 It is the effect diagram after splicing of the present invention. DETAILED DESCRIPTION

[0032] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features, and effects of the technical solutions proposed by the present invention. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0033] In one embodiment of the present invention, referring to Figure 1-Figure 7 , provides a method for stitching infrared images of non-uniform speed vehicles based on temperature difference features, including the following steps:

[0034] Step 100: When a vehicle passes through a security inspection channel, an infrared image of the vehicle is captured in a single frame manner;

[0035] Step 200: setting a ROI region on each frame of vehicle infrared image;

[0036] Step 300: Based on the ROI region of each frame of the vehicle infrared image, a column feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference, and similar columns in adjacent vehicle infrared images are matched based on the column feature matrix, i.e., the columns to be spliced;

[0037] Step 400: Based on the ROI region of each frame of the vehicle infrared image, a row feature matrix of the ROI region of each frame of the vehicle infrared image is calculated by normalizing the temperature difference, and similar rows in adjacent vehicle infrared images are matched based on the row feature matrix, i.e., rows to be spliced;

[0038] Step 500: stitching two adjacent frames of vehicle infrared images based on the columns to be stitched and the rows to be stitched;

[0039] Step 600: smoothing the joint seams between two adjacent frames of vehicle infrared images to obtain a final joint image.

[0040] The following is a detailed explanation of each of the above steps:

[0041] Step 100: When a vehicle passes through a security inspection channel, an infrared image of the vehicle is captured in a single-frame manner.

[0042] When a vehicle passes through the security inspection channel, the medium-wave infrared array camera is activated to capture infrared images of the vehicle in single frames. The captured infrared images of the vehicle are stored in a sequence in chronological order.

[0043] Step 200: Set a ROI region on each frame of vehicle infrared image.

[0044] With the installation point of the medium wave infrared array camera as the origin, the vehicle's driving direction as the X axis, and the vertical direction as the Y axis, the camera coordinate system is established, and the ROI area is set on each frame of the vehicle infrared image. Figure 2 , the ROI area range is (x1, y1) to (x2, y2) to reduce the amount of calculation.

[0045] ROI area selection principle: Cover the characteristic areas that may appear on the side or top of the car body, such as the door edge, window outline, roof seam, etc., and avoid selecting areas with uniform temperature and no characteristic changes, such as solid color paint.

[0046] Step 300: Based on the ROI area of ​​each frame of vehicle infrared image, calculate the column feature matrix of the ROI area of ​​each frame of vehicle infrared image by normalizing the temperature difference, and match similar columns in adjacent vehicle infrared images based on the column feature matrix, that is, the columns to be spliced.

[0047] As an example, this step 300 may include the following sub-steps:

[0048] Step 310: Calculate the column average temperature of the ROI area of ​​each frame of the vehicle infrared image , as the column reference temperature :

[0049] ;

[0050] In the above formula, Indicates the i No. j The temperature value of each pixel in the ROI area is represented by N.

[0051] Step 320: Based on the column reference temperature, calculate the normalized temperature difference for each column of the ROI region of each frame of the vehicle infrared image to obtain a column feature matrix.

[0052] For each pixel in each column of the vehicle infrared image ROI area, calculate the difference between it and the column reference temperature :

[0053] ;

[0054] Based on the difference in column reference temperatures, a minimum temperature difference in the vehicle infrared image ROI region column and a maximum temperature difference in the vehicle infrared image ROI region column are obtained; and normalization processing is performed based on the minimum temperature difference in the vehicle infrared image ROI region column and the maximum temperature difference in the vehicle infrared image ROI region column:

[0055] ;

[0056] In the above formula, represents the normalized column difference; Indicates the maximum temperature difference in the ROI area of ​​the vehicle infrared image; Indicates the minimum temperature difference of the ROI area in the vehicle infrared image.

[0057] Through normalization, the temperature change of each column is converted into relative difference, reducing the influence of absolute temperature value on feature matching.

[0058] Step 330: Using the cross-correlation coefficient or Euclidean distance, calculate the similarity of the column feature matrices of the ROI region of two adjacent vehicle infrared image frames, and find the column with the highest similarity in the two adjacent vehicle infrared image frames, which is the column to be spliced.

[0059] The column to be stitched of the k-1th frame of vehicle infrared image is c1, and the column to be stitched of the kth frame of vehicle infrared image is c2.

[0060] Step 400: Based on the ROI area of ​​each frame of vehicle infrared image, calculate the row feature matrix of the ROI area of ​​each frame of vehicle infrared image by normalizing the temperature difference, and match similar rows in adjacent vehicle infrared images based on the row feature matrix, that is, the rows to be spliced.

[0061] As an example, this step 400 may include the following sub-steps:

[0062] Step 410: Calculate the row average temperature of the ROI region of each frame of the vehicle infrared image , as the row reference temperature :

[0063] ;

[0064] In the above formula, Indicates the i No. j The temperature value of each pixel in the ROI area is represented by N.

[0065] Step 420: Based on the row reference temperature, for each row of the ROI region of each frame of the vehicle infrared image, a normalized temperature difference is calculated to obtain a row feature matrix.

[0066] For each pixel in each column of the vehicle infrared image ROI area, calculate the difference between it and the row reference temperature :

[0067] ;

[0068] Based on the difference in the row reference temperatures, a minimum temperature difference in the vehicle infrared image ROI region and a maximum temperature difference in the vehicle infrared image ROI region are obtained; and normalization processing is performed based on the minimum temperature difference in the vehicle infrared image ROI region and the maximum temperature difference in the vehicle infrared image ROI region:

[0069] ;

[0070] In the above formula, represents the normalized row difference; Indicates the maximum temperature difference in the ROI area of ​​the vehicle infrared image; Indicates the minimum temperature difference in the ROI area of ​​the vehicle infrared image.

[0071] Step 430: Using the Euclidean distance of the mutual correlation coefficient, calculate the similarity of the row feature matrix of the ROI region of the two adjacent vehicle infrared image frames, and find the row with the highest similarity in the two adjacent vehicle infrared image frames, which is the row to be spliced.

[0072] The line to be stitched of the k-1th frame of vehicle infrared image is r1, and the line to be stitched of the kth frame of vehicle infrared image is r2.

[0073] Step 500: stitching two adjacent frames of vehicle infrared images based on the columns to be stitched and the rows to be stitched.

[0074] Align column c2 and its right area in frame k with column c1 and its right area in frame k−1. Align row r2 and its lower area in frame k with row r1 and its lower area in frame k−1. Determine the final seam position by primarily stitching columns together, supplemented by stitching rows. The midpoint between column seams c1 and c2, or between row seams r1 and r2, can be selected as the final seam.

[0075] Step 600: Using a hybrid transition method, smooth the seams between two adjacent frames of vehicle infrared images to obtain a final stitched image.

[0076] For the stitching seams, there may be obvious steps. The hybrid transition method is used to eliminate the difference at the stitching seams as much as possible to obtain the final stitching image.

[0077] The hybrid transition method includes: selecting a certain number of columns (e.g., two columns on each side) on the left and right sides of the seam as smoothing areas according to the specific location of the seam; applying a smoothing algorithm, such as linear interpolation or Gaussian filtering, to the selected smoothing areas; and fusing the smoothed columns with the unsmoothed areas to obtain the final stitched image.

[0078] Alternatively, a mixed transition method may be used, comprising selecting the center point of the k-1 frame splicing location as a reference temperature point and recording the temperature value of the point; finding a position corresponding to the k-1 frame reference temperature point in the k-1 frame in the k-th frame, and calculating the temperature difference between the two frames at the position; subtracting the calculated temperature difference from the temperatures of all points in the k-th frame to make the temperature of the k-th frame closer to that of the k-1 frame at the splicing location; and splicing the adjusted k-th frame image with the k-1 frame image to obtain a final spliced ​​image.

[0079] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for stitching infrared images of vehicles with non-uniform speed based on temperature difference features, characterized in that: The following steps are involved: When a vehicle passes through the security inspection channel, a single-frame infrared image of the vehicle is captured. A ROI region is set on each frame of the vehicle infrared image. The ROI region selection principle is to cover the characteristic areas on the side or top of the vehicle body, including the door edges, window outlines, and roof seams, and avoid areas with uniform temperature and no characteristic changes. Based on the ROI area of ​​each frame of vehicle infrared image, the column feature matrix of the ROI area of ​​each frame of vehicle infrared image is calculated by normalizing the temperature difference. Based on the column feature matrix, similar columns in adjacent vehicle infrared images are matched, which are the columns to be spliced, including: Calculate the average column temperature of the ROI region of each frame of the vehicle infrared image as the column reference temperature; based on the column reference temperature, calculate the normalized temperature difference for each column of the ROI region of each frame of the vehicle infrared image to obtain a column feature matrix; calculate the similarity of the column feature matrices of the ROI regions of two adjacent vehicle infrared image frames, and find the column with the highest similarity between the two adjacent vehicle infrared image frames, which is the column to be spliced; Based on the ROI area of ​​each frame of vehicle infrared image, the row feature matrix of the ROI area of ​​each frame of vehicle infrared image is calculated by normalizing the temperature difference, and similar rows in adjacent vehicle infrared images are matched based on the row feature matrix, which are the rows to be spliced, including: Calculate the average row temperature of the ROI region of each frame of the vehicle infrared image as the row reference temperature; based on the row reference temperature, calculate the normalized temperature difference for each row of the ROI region of each frame of the vehicle infrared image to obtain a row feature matrix; calculate the similarity of the row feature matrices of the ROI regions of two adjacent vehicle infrared image frames, and find the row with the highest similarity between the two adjacent vehicle infrared image frames, which is the row to be spliced; Based on the columns to be spliced ​​and the rows to be spliced, two adjacent frames of vehicle infrared images are spliced; and the splicing seams of the two adjacent frames of vehicle infrared images are smoothed to obtain a final spliced ​​image.

2. The method for stitching infrared images of vehicles with non-uniform speed based on temperature difference characteristics according to claim 1, characterized in that: For each frame of vehicle infrared image, a ROI area is defined within a predefined fixed coordinate range.

3. The method for stitching infrared images of vehicles with non-uniform speed based on temperature difference characteristics according to claim 1, characterized in that: The similarity of the column feature matrices of the ROI regions of two adjacent vehicle infrared image frames is calculated using the mutual correlation coefficient or Euclidean distance.

4. The method for stitching infrared images of vehicles with non-uniform speed based on temperature difference characteristics according to claim 1, characterized in that: The similarity of the row feature matrices of the ROI regions of two adjacent vehicle infrared image frames is calculated using the cross-correlation coefficient or Euclidean distance.

5. The method for stitching infrared images of vehicles with non-uniform speed based on temperature difference characteristics according to claim 1, characterized in that: Smooth the seams between two adjacent frames of vehicle infrared images to obtain the final stitched image, including: The specific position of the seam is determined according to the columns and rows to be spliced; a preset number of columns are selected on both sides of the seam as the area to be smoothed; and the area to be smoothed is smoothed to obtain the final spliced ​​image.

Images