Non-uniform vehicle infrared image splicing method based on temperature difference characteristics

Abstract

The invention belongs to the technical field of image splicing, and particularly relates to a non-uniform vehicle infrared image splicing method based on temperature difference characteristics. When a vehicle passes through the security check channel, acquiring a vehicle infrared image in a single frame mode; setting an ROI (Region of Interest) on each frame of vehicle infrared image; on the basis of the ROI of each frame of vehicle infrared image, calculating a column feature matrix of the ROI of each frame of vehicle infrared image through normalized temperature difference, and matching similar columns in adjacent frames of vehicle infrared images on the basis of the column feature matrix, namely columns to be spliced; on the basis of the ROI of each frame of vehicle infrared image, calculating a row feature matrix of the ROI of each frame of vehicle infrared image through normalized temperature difference, and matching similar rows in adjacent vehicle infrared images on the basis of the row feature matrix, namely rows to be spliced; based on the to-be-spliced columns and rows, two adjacent frames of vehicle infrared images are spliced; and smoothing the splicing seam of the two adjacent frames of vehicle infrared images so as to obtain a final spliced image.

Description

Technical Field

[0001] The invention belongs to the technical field of image stitching, and in particular relates to a method for stitching infrared images of non-uniform-speed vehicles based on temperature difference features. Background Art

[0002] In the vehicle security inspection scenario, collecting temperature information from multiple perspectives (left, right, and top) of vehicles passing through the security inspection channel is a key technical means to ensure public safety and prevent safety hazards such as the transportation of dangerous goods. However, the existing technology has the following problems: On the one hand, the length of the vehicle under test (such as buses, trucks, etc.) usually exceeds the imaging range of a single-frame infrared image. Due to the resolution and lens viewing angle limitations of medium-wave infrared array cameras, a single-frame image can only capture a local area of the vehicle body, such as the front, rear or part of the side, making it impossible to directly obtain temperature information of the entire vehicle. If it relies on simple inter-frame superposition or splicing, data loss or redundancy may occur due to factors such as body occlusion and viewing angle changes, affecting the integrity of the temperature field distribution.

[0003] On the other hand, vehicles are usually in a non-uniform driving state in the security inspection channel, and their speed fluctuations will cause significant position offsets, rotations or scaling differences between multiple frames. This dynamic change further increases the complexity of reconstructing the temperature field of the entire vehicle body: infrared images are characterized by blurred edges, and traditional stitching algorithms based on static scenes, such as SIFT and ORB based on feature points, are difficult to effectively handle dynamic targets and targets with blurred edges, and are prone to stitching faults, ghosting or geometric distortion; while stitching methods based on global optimization, such as optical flow methods and variational models, can improve accuracy, but the computational complexity is high and it is difficult to meet the real-time requirements at a 25Hz frame rate.

[0004] In addition, infrared images have low contrast and little texture information, and it is difficult to match the features of the vehicle body surface (such as smooth metal and paint), which further limits the applicability of existing stitching technology. Therefore, how to break through the limitation of single-frame imaging, overcome the interference of non-uniform motion, and achieve accurate matching and stitching of vehicle body surface features has become a technical problem that needs to be solved in the current field of vehicle security infrared detection. Summary of the invention

[0005] In view of the problems in the prior art such as insufficient single-frame imaging range, large non-uniform motion interference, and difficult infrared image feature matching, the present invention provides a non-uniform vehicle infrared image stitching method based on temperature difference features to achieve reconstruction of the overall temperature field of the vehicle.

[0006] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides a method for splicing infrared images of non-uniform speed vehicles based on temperature difference characteristics, comprising the following steps: When a vehicle passes through the security inspection channel, the vehicle infrared image is collected in a single-frame manner; an ROI region is set on each frame of the vehicle infrared image; Based on the ROI region of each frame of the vehicle infrared image, the column feature matrix of the ROI region of each frame of the vehicle infrared image is calculated through normalized temperature difference, and the similar columns in adjacent vehicle infrared images are matched based on the column feature matrix, which are the columns to be stitched; Based on the ROI region of each frame of the vehicle infrared image, the row feature matrix of the ROI region of each frame of the vehicle infrared image is calculated through normalized temperature difference, and the similar rows in adjacent vehicle infrared images are matched based on the row feature matrix, which are the rows to be stitched; Based on the columns to be stitched and the rows to be stitched, two adjacent frames of vehicle infrared images are stitched; the stitching seam of two adjacent frames of vehicle infrared images is smoothed to obtain the final stitched image.

[0007] Further, for each frame of the vehicle infrared image, the ROI region is delimited within a predefined fixed coordinate range.

[0008] Further, the method of calculating the column feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference based on the ROI region of each frame of the vehicle infrared image, and matching the similar columns in adjacent vehicle infrared images based on the column feature matrix, which are the columns to be stitched, includes: Calculating the column average temperature of the ROI region of each frame of the vehicle infrared image as the column reference temperature; based on the column reference temperature, calculating the normalized temperature difference for each column of the ROI region of each frame of the vehicle infrared image to obtain the column feature matrix; calculating the similarity of the column feature matrices of the ROI regions of two adjacent frames of vehicle infrared images, and finding the column with the highest similarity in two adjacent frames of vehicle infrared images, which is the column to be stitched.

[0009] Further, the similarity of the column feature matrices of the ROI regions of two adjacent frames of vehicle infrared images is calculated using the cross-correlation coefficient or the Euclidean distance.

[0010] Further, the method of calculating the row feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference based on the ROI region of each frame of the vehicle infrared image, and matching the similar rows in adjacent vehicle infrared images based on the row feature matrix, which are the rows to be stitched, includes: Calculating the row average temperature of the ROI region of each frame of the vehicle infrared image as the row reference temperature; based on the row reference temperature, calculating the normalized temperature difference for each row of the ROI region of each frame of the vehicle infrared image to obtain the row feature matrix; using the cross-correlation coefficient or the Euclidean distance to calculate the similarity of the row feature matrices of the ROI regions of two adjacent frames of vehicle infrared images, and finding the row with the highest similarity in two adjacent frames of vehicle infrared images, which is the row to be stitched.

[0011] Further, the similarity of the row feature matrices of the ROI regions of adjacent two-frame vehicle infrared images is calculated using the cross-correlation coefficient or Euclidean distance.

[0012] Further, the splicing seam of adjacent two-frame vehicle infrared images is smoothed to obtain the final spliced image, including: Determine the specific position of the splicing seam according to the columns and rows to be spliced; select a preset number of columns on both the left and right sides of the splicing seam as the regions to be smoothed; perform smoothing processing on the regions to be smoothed to obtain the final spliced image.

[0013] Compared with the prior art, the present invention has the following technical effects: The present invention provides a non-uniform vehicle infrared image splicing method based on temperature difference features. By extracting the temperature difference features of infrared images, precise matching and splicing of adjacent frame images are achieved, non-uniform motion interference is eliminated, and a spliced image with natural transition is obtained. This method has the advantages of wide imaging range, high matching accuracy, and natural splicing effect, and is applicable to fields such as vehicle security inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 is a flowchart of the present invention; Figure 2 is a schematic diagram of the ROI region of a single-frame image of the present invention; Figure 3 is a partial first single-frame image to be spliced in the present invention; Figure 4 is a partial second single-frame image to be spliced in the present invention; Figure 5 is a partial third single-frame image to be spliced in the present invention; Figure 6 is a partial fourth single-frame image to be spliced in the present invention; Figure 7 is the effect diagram after splicing of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes the implementation manner, structure, features and effects of the technical solution proposed according to the present invention in detail with reference to the accompanying drawings and preferred embodiments. Specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0017] In one embodiment of the present invention, referring to Figures 1 - 7 , a non-uniform vehicle infrared image stitching method based on temperature difference features is provided, including the following steps: Step 100: When the vehicle passes through the security inspection channel, collect the vehicle infrared image in a single-frame manner; Step 200: Set the ROI area on each frame of the vehicle infrared image; Step 300: Based on the ROI area of each frame of the vehicle infrared image, calculate the column feature matrix of the ROI area of each frame of the vehicle infrared image through normalized temperature difference, and match the similar columns in the adjacent vehicle infrared images based on the column feature matrix, which are the columns to be stitched; Step 400: Based on the ROI area of each frame of the vehicle infrared image, calculate the row feature matrix of the ROI area of each frame of the vehicle infrared image through normalized temperature difference, and match the similar rows in the adjacent vehicle infrared images based on the row feature matrix, which are the rows to be stitched; Step 500: Stitch the adjacent two frames of vehicle infrared images based on the columns to be stitched and the rows to be stitched; Step 600: Smooth the stitching seam of the adjacent two frames of vehicle infrared images to obtain the final stitched image.

[0018] The following details each of the above steps: Step 100: When the vehicle passes through the security inspection channel, collect the vehicle infrared image in a single-frame manner.

[0019] When the vehicle passes through the security inspection channel, start the mid-wave infrared area array camera and collect the vehicle infrared image in a single-frame manner. Store the collected vehicle infrared images in a sequence in chronological order.

[0020] Step 200: Set the ROI area on each frame of the vehicle infrared image.

[0021] Taking the installation point of the mid-wave infrared area array camera as the origin, the vehicle driving direction as the X-axis, and the vertical direction as the Y-axis, establish a camera coordinate system, and set the ROI area on each frame of the vehicle infrared image. Referring to Figure 2 , the range of the ROI area is from (x1, y1) to (x2, y2) to reduce the calculation amount.

[0022] ROI region selection principle: Cover the possible feature regions on the side or top of the vehicle body, such as door edges, window contours, roof seams, etc., and avoid selecting regions with uniform temperature and no feature changes, such as solid-color vehicle paint.

[0023] Step 300: Based on the ROI region of each frame of vehicle infrared image, calculate the column feature matrix of the ROI region of each frame of vehicle infrared image through normalized temperature difference, and match the similar columns in the adjacent vehicle infrared images based on the column feature matrix, which are the columns to be stitched.

[0024] As an example, this step 300 may include the following sub-steps: Step 310: Calculate the column average temperature of the ROI region of each frame of vehicle infrared image , as the column reference temperature : ; In the above formula, represents the temperature value of the i th pixel in the j th column; N represents the total number of pixels in each column of the ROI region.

[0025] Step 320: Based on the column reference temperature, calculate the normalized temperature difference for each column of the ROI region of each frame of vehicle infrared image to obtain the column feature matrix.

[0026] For each pixel in each column of the ROI region of the vehicle infrared image, calculate the difference between it and the column reference temperature : ; Based on the difference of the column reference temperature, obtain the minimum column temperature difference and the maximum column temperature difference of the ROI region of the vehicle infrared image; and perform normalization processing based on the minimum column temperature difference and the maximum column temperature difference of the ROI region of the vehicle infrared image: ; In the above formula, represents the normalized column difference; represents the maximum column temperature difference of the ROI region of the vehicle infrared image; represents the minimum column temperature difference of the ROI region of the vehicle infrared image.

[0027] Through normalization processing, convert the temperature change of each column into a relative difference, reducing the influence of the absolute temperature value on feature matching.

[0028] Step 330: Use the cross-correlation coefficient or Euclidean distance to calculate the similarity of the column feature matrices of the ROI regions of two adjacent frames of vehicle infrared images, and find the column with the highest similarity in the two adjacent frames of vehicle infrared images, which is the column to be stitched.

[0029] The column to be stitched of the vehicle infrared image in the (k - 1)-th frame is c1, and the column to be stitched of the vehicle infrared image in the k-th frame is c2.

[0030] Step 400: Based on the ROI region of each frame of the vehicle infrared image, calculate the row feature matrix of the ROI region of each frame of the vehicle infrared image through the normalized temperature difference, and match the similar rows in the adjacent vehicle infrared images based on the row feature matrix, which are the rows to be stitched.

[0031] As an example, this step 400 may include the following sub-steps: Step 410: Calculate the row average temperature of the ROI region of each frame of the vehicle infrared image as the row reference temperature : ; In the above formula, represents the temperature value of the i -th pixel in the j -th column; N represents the total number of pixels in each column of the ROI region.

[0032] Step 420: Based on the row reference temperature, calculate the normalized temperature difference for each row of the ROI region of each frame of the vehicle infrared image to obtain the row feature matrix.

[0033] For each pixel in each column of the ROI region of the vehicle infrared image, calculate the difference between it and the row reference temperature : ; Based on the difference of the row reference temperature, obtain the minimum row temperature difference and the maximum row temperature difference of the ROI region of the vehicle infrared image; and perform normalization processing based on the minimum row temperature difference and the maximum row temperature difference of the ROI region of the vehicle infrared image: ; In the above formula, represents the normalized row difference; represents the maximum row temperature difference of the ROI region of the vehicle infrared image; represents the minimum row temperature difference of the ROI region of the vehicle infrared image.

[0034] Step 430: Use the Euclidean distance of the cross-correlation coefficient to calculate the similarity of the row feature matrices of the ROI regions of two adjacent frames of the vehicle infrared image, and find the row with the highest similarity in the two adjacent frames of the vehicle infrared image, which is the row to be stitched.

[0035] The row to be stitched of the vehicle infrared image in the (k - 1)-th frame is r1, and the row to be stitched of the vehicle infrared image in the k-th frame is r2.

[0036] Step 500: Stitch two adjacent frames of vehicle infrared images based on the columns and rows to be stitched.

[0037] Align the column c2 and its right region of the k-th frame with the column c1 and its right region of the (k - 1)-th frame; align the row r2 and its lower region of the k-th frame with the row r1 and its lower region of the (k - 1)-th frame; mainly perform column stitching and supplementarily perform row stitching to determine the position of the final stitching seam. The midpoint of the column stitching seams c1 and c2, or the midpoint of the row stitching seams r1 and r2, can be selected as the final stitching seam.

[0038] Step 600: Smooth the stitching seam of two adjacent frames of vehicle infrared images by using a hybrid transition method to obtain the final stitched image.

[0039] For the stitching seam, it may cause obvious steps. Use the hybrid transition method to eliminate the difference at the stitching as much as possible to obtain the final stitched image. The hybrid transition method includes: according to the specific position of the stitching seam; select a certain number of columns (such as two columns on each side) on both the left and right sides of the stitching seam as the smoothing regions; apply a smoothing algorithm, such as linear interpolation or Gaussian filtering, to the selected smoothing regions; fuse the smoothed columns with the unsmoothed regions to obtain the final stitched image.

[0040] Or, the hybrid transition method includes selecting the center point at the stitching of the (k - 1)-th frame as the reference temperature point and recording the temperature value at this point; in the k-th frame, find the corresponding position of the reference temperature point of the (k - 1)-th frame and calculate the temperature difference between the two frames at this position; subtract the calculated temperature difference from the temperature of all points in the k-th frame to make the temperature of the k-th frame closer to that of the (k - 1)-th frame at the stitching; stitch the adjusted k-th frame image and the (k - 1)-th frame image to obtain the final stitched image.

[0041] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A non-uniform vehicle infrared image stitching method based on temperature difference characteristics, characterized in that, Including the following steps: When the vehicle passes through the security inspection channel, collect the vehicle infrared image in a single-frame manner; set the ROI region on each frame of the vehicle infrared image; Based on the ROI region of each frame of the vehicle infrared image, calculate the column feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference, and match the similar columns in the adjacent vehicle infrared images based on the column feature matrix, which are the columns to be spliced; Based on the ROI region of each frame of the vehicle infrared image, calculate the row feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference, and match the similar rows in the adjacent vehicle infrared images based on the row feature matrix, which are the rows to be spliced; Based on the columns to be spliced and the rows to be spliced, splice the adjacent two frames of vehicle infrared images; smooth the splicing seam of the adjacent two frames of vehicle infrared images to obtain the final spliced image.

2. The non-uniform vehicle infrared image stitching method based on temperature difference features according to claim 1, characterized in that For each frame of the vehicle infrared image, delimit the ROI region within the predefined fixed coordinate range.

3. A non-uniform vehicle infrared image stitching method based on temperature difference characteristics according to claim 1, characterized in that, The step of calculating the column feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference and matching the similar columns in the adjacent vehicle infrared images based on the column feature matrix, which are the columns to be spliced, includes: Calculate the column average temperature of the ROI region of each frame of the vehicle infrared image as the column reference temperature; based on the column reference temperature, calculate the normalized temperature difference for each column of the ROI region of each frame of the vehicle infrared image to obtain the column feature matrix; calculate the similarity of the column feature matrices of the ROI regions of the adjacent two frames of vehicle infrared images, and find the column with the highest similarity in the adjacent two frames of vehicle infrared images, which is the column to be spliced.

4. A non-uniform vehicle infrared image stitching method based on temperature difference characteristics according to claim 3, characterized in that Use the cross-correlation coefficient or Euclidean distance to calculate the similarity of the column feature matrices of the ROI regions of the adjacent two frames of vehicle infrared images.

5. A non-uniform vehicle infrared image stitching method based on temperature difference features according to claim 1, characterized in that, The step of calculating the row feature matrix of the ROI region of each frame of the vehicle infrared image through normalized temperature difference and matching the similar rows in the adjacent vehicle infrared images based on the row feature matrix, which are the rows to be spliced, includes: Calculate the row average temperature of the ROI region of each frame of the vehicle infrared image as the row reference temperature; based on the row reference temperature, calculate the normalized temperature difference for each row of the ROI region of each frame of the vehicle infrared image to obtain the row feature matrix; calculate the similarity of the row feature matrices of the ROI regions of the adjacent two frames of vehicle infrared images, and find the row with the highest similarity in the adjacent two frames of vehicle infrared images, which is the row to be spliced.

6. A non-uniform vehicle infrared image stitching method based on temperature difference characteristics according to claim 5, wherein, Use the cross-correlation coefficient or Euclidean distance to calculate the similarity of the row feature matrices of the ROI regions of the adjacent two frames of vehicle infrared images.

7. A non-uniform vehicle infrared image stitching method based on temperature difference characteristics according to claim 1, characterized in that The step of smoothing the splicing seam of the adjacent two frames of vehicle infrared images to obtain the final spliced image includes: Determine the specific position of the splicing seam according to the columns to be spliced and the rows to be spliced; select a preset number of columns on both the left and right sides of the splicing seam as the regions to be smoothed; perform smoothing processing on the regions to be smoothed to obtain the final spliced image.

Images