Infrared image non-overlapping splicing method and device

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an infrared image non-overlapping splicing method and device. The traditional image splicing technology is a technology for splicing a plurality of frames of images with overlapped parts, including images obtained at different time, at different visual angles or by different sensors, into a seamless panorama or a high-resolution image. The invention provides a non-overlapping image splicing method and device. According to the infrared image non-overlapping splicing method, continuous multiple frames of infrared images are spliced into a panorama, the scene is continuous, breakage and repetition are avoided, the brightness is smooth, and splicing seams are avoided. The brightness curve does not need to be adjusted manually. The method is simple in calculation and high in adaptability.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method and a device for non-overlapping splicing of infrared images. Background Art

[0002] The difference between panoramic scanning monitoring equipment and traditional turntable systems is that the rotation speed is faster, reaching ≥180rps. In order to avoid imaging smear, a fast-reflection mirror is used to compensate for each frame of the imaging process, and then stitch them into a panoramic image. The traditional stitching method generally stitches two or more images with overlapping areas together by feature matching, and transforms images taken from multiple different perspectives to the same perspective to stitch them into a wide-view image. The basic process of image stitching includes (1) taking multiple / sequence images of a certain scene; (2) calculating the transformation relationship between the second image and the first image; (3) adding the second image to the coordinate system of the first image; (4) fusion / synthesis after transformation; (5) repeating the above process in multiple image scenes. The whole process is computationally complex and time-consuming, which is not conducive to embedding and transplantation. If the scene is distorted, ghost images are likely to appear in the fused area.

[0003] Invention patent CN202310158914.X discloses a panoramic image stitching method, system and stitching terminal without overlapping areas, which relates to the field of infrared imaging. The captured image information is obtained to form an image sequence; the image information in the image sequence is arranged in a preset order; the image is converted into a grayscale image; the brightness difference between the right edge of the previous image and the left edge of the next image is counted as the initial threshold for adjusting the brightness; the image is preprocessed to define the brightness adjustment curve of the image; according to the brightness of the image from dark to bright from top to bottom, and combined with the initial threshold for adjusting the brightness, the images in the image sequence are stitched to form a panoramic image. The present invention uses the center of the image as a reference point and adjusts the image brightness of the border around the image in combination with the edge brightness of two adjacent images, so that the brightness of the image connection is similar after the image is stitched, which can meet the actual use requirements.

[0004] The invention method has the following problems in the application process of splicing two adjacent frames of scene images: (1) If the brightness at the joint is uneven or there are boundaries of objects in the scene, such as buildings, electric poles, and other objects with a large temperature difference from the background, the brightness at the joint cannot be consistent; (2) The method of using the center of the image as the reference point and the diameter of the Gaussian circle as the distance to define the image brightness adjustment curve ignores the actual brightness difference at the joint of adjacent frames and cannot guarantee the brightness consistency of the joint after splicing; (3) The method of defining the image brightness curve and adjusting the brightness curve requires manual intervention and has limited control effect. Therefore, when this method is actually used in equipment such as weekly scanning light mines, it is difficult to meet the requirement of completely consistent brightness at the joint of adjacent frames. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to propose a method and device for stitching images without overlapping, so that no brightness difference is generated at the boundary of objects at the seam, and no manual adjustment of the brightness curve is required, so as to obtain a panoramic image with completely consistent brightness of objects at the seam.

[0006] The device of the present invention is composed of three parts: an image acquisition module, an image processing module, and an image display module. The image acquisition module includes a horizontal scanning module, a fast response compensation module, an optical lens module, and a camera; the image processing module includes an image cropping module and an image stitching module; and the image display module includes a panoramic image display module.

[0007] The workflow of the device of the present invention comprises the following steps: (1) Image acquisition: Acquisition of a circular scanning photoelectric image sequence after fast response compensation; (2) Image cropping: Through the initial image stitching, determine how many columns of pixels to crop to meet the stitching panorama requirements; (3) Image stitching: Perform brightness statistical analysis on the stitching of adjacent frames without overlap, adjust them, and then stitch them together; (4) Image display: Display the stitched panoramic image.

[0008] The infrared image non-overlapping stitching method of the present invention comprises the following steps: (1) Receive the acquired continuous image sequence; (2) Sorting frames and confirming the left and right images for splicing adjacent frames; (3) Calculate the brightness mean nGrayLeftRM(i) of the nNum pixels in each row of the right border of the left image; calculate the brightness mean nGrayRightLM(i) of the nNum pixels in each row of the left border of the right image; where i is the row number; (4) Calculate the brightness adjustment target for each row nGrayGoal(i) = (nGrayLeftRM(i)+nGrayRightLM(i)) / 2; (5) Calculate the brightness correction value of each row of pixels on the right border of the left image: nGrayCorrectLeft(i) = nGryaGoal(i) - nGrayLeftRM(i); Calculate the brightness correction value of each row of pixels on the left border of the right image: nGrayCorrectRight(i) = nGryaGoal(i) - nGrayRightLM(i); (6) Use the adjacent rows of the left image to smooth the brightness correction value of each row of pixels at the right edge of the left image, nGrayCorrectLeft(i), to obtain nGrayCorrectLeftNew(i); use the adjacent rows of the right image to smooth the brightness correction value of each row of pixels at the left edge of the right image, nGrayCorrectRight(i), to obtain nGrayCorrectRightNew(i); (7) The brightness of each line on the right edge of the left image is nonlinearly smoothed according to a window of a certain width nWinWidthLeft; the brightness of each line on the left edge of the right image is nonlinearly smoothed according to a window of a certain width nWinWidthRight; (8) The left and right images are pixel-stitched to complete the stitching of two adjacent frames of images; sequential image stitching is performed in sequence to generate a panoramic image.

[0009] Traditional image stitching technology is a technology that stitches together several frames of overlapping images, including images obtained at different times, different viewing angles or different sensors, into a seamless panorama or high-resolution image. The present invention proposes a non-overlapping image stitching method and device. The infrared image non-overlapping stitching method of the present invention stitches together multiple consecutive infrared images into a panorama with continuous, unbroken and non-repetitive scenes and smooth brightness without stitching. There is no need to manually adjust the brightness curve, the method is simple to calculate and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a module diagram of the device of the present invention.

[0011] Figure 2 It is the working flow chart of the device of the present invention.

[0012] Figure 3 This is a flow chart of the non-overlapping image panoramic stitching algorithm of the present invention.

[0013] Figure 4 It is the effect of direct stitching of images without overlapping.

[0014] Figure 5 This is the splicing effect of the method of the present invention. DETAILED DESCRIPTION

[0015] The device of the embodiment of the present invention is composed of three parts: an image acquisition module, an image processing module, and an image display module, and can acquire 360-degree panoramic images. The image acquisition module includes a horizontal scanning module, a fast response compensation module, an optical lens module, and a camera; the image processing module includes an image cropping module and an image stitching module; and the image display module includes a panoramic image display module. The device workflow includes the following steps: (1) Image acquisition. Collect the photoelectric image sequence after fast response compensation. Each cycle is 1 second, and 50 frames of images are collected. The resolution of each frame is 640 width × 512 height. (2) Image cropping. After the initial image stitching, determine how many columns of pixels to crop to meet the requirements of the stitching panorama. If necessary, crop 16 columns on each side of each frame of image, and the resolution of each frame of image is 608 width × 512 height; (3) Image stitching. Perform brightness statistical analysis on the stitching of adjacent frames without overlap, make adjustments, and then stitch them together; (4) Image display: The stitched panoramic image is displayed with a width of 30400 and a height of 512.

[0016] The present invention stitches multiple consecutive infrared images into a panoramic image, with continuous, unbroken, non-repetitive scenes and smooth brightness without seams. The stitching method includes the following steps: (1) Receive and collect continuous image sequences at 50 frames per second; (2) Sorting frames and confirming the left and right images for splicing adjacent frames; (3) Calculate the brightness mean nGrayLeftRM(i) of nNum=5 pixels in each row of the right border of the left image; calculate the brightness mean nGrayRightLM(i) of nNum=5 pixels in each row of the left border of the right image; where i is the row number; (4) Calculate the brightness adjustment target for each row nGrayGoal(i) = (nGrayLeftRM(i)+nGrayRightLM(i)) / 2; (5) Calculate the brightness correction value of each row of pixels on the right border of the left image: nGrayCorrectLeft(i) = nGryaGoal(i) - nGrayLeftRM(i); Calculate the brightness correction value of each row of pixels on the left border of the right image: nGrayCorrectRight(i) = nGryaGoal(i) - nGrayRightLM(i); (6) Use the adjacent rows to smooth the brightness correction value of each row of pixels on the right edge of the left image, nGrayCorrectLeft(i), to obtain nGrayCorrectLeftNew(i); use the adjacent rows to smooth the brightness correction value of each row of pixels on the left edge of the right image, nGrayCorrectRight(i), to obtain nGrayCorrectRightNew(i); (7) The brightness of each line on the right edge of the left image is nonlinearly smoothed according to a certain width window nWinWidthLeft=100pixel; the brightness of each line on the left edge of the right image is nonlinearly smoothed according to a certain width window nWinWidthRight=100pixel. Here, a quadratic curve is used for nonlinear brightness smoothing; (8) The left and right images are pixel-stitched to complete the stitching of two adjacent frames of images; sequential image stitching is performed in sequence to generate a panoramic image.

[0017] The key points of the present invention are: (1) The brightness adjustment target of the left and right seams of adjacent frames is consistent, and there is no brightness difference; (2) In the special case where there is an imaging target edge at the seam, smoothing of adjacent brightness targets is performed; (3) The transition method of the image brightness adjustment range adopts nonlinear brightness transition, which does not add brightness adjustment traces inside the image and makes the brightness transition more natural.

Claims

1. A method for non-overlapping mosaic of infrared images, characterized in that: It includes the following steps: (1) Receive the acquired continuous image sequence; (2) Sorting frames and confirming the left and right images for splicing adjacent frames; (3) Calculate the brightness mean nGrayLeftRM(i) of the nNum pixels in each row of the right border of the left image; calculate the brightness mean nGrayRightLM(i) of the nNum pixels in each row of the left border of the right image; Where i is the row number; (4) Calculate the brightness adjustment target for each row nGrayGoal(i) = (nGrayLeftRM(i)+nGrayRightLM(i)) / 2; (5) Calculate the brightness correction value of each row of pixels on the right border of the left image: nGrayCorrectLeft(i) = nGryaGoal(i) - nGrayLeftRM(i); Calculate the brightness correction value of each row of pixels on the left border of the right image: nGrayCorrectRight(i) = nGryaGoal(i) - nGrayRightLM(i); (6) Use the adjacent rows of the left image to smooth the brightness correction value of each row of pixels at the right edge of the left image, nGrayCorrectLeft(i), to obtain nGrayCorrectLeftNew(i); use the adjacent rows of the right image to smooth the brightness correction value of each row of pixels at the left edge of the right image, nGrayCorrectRight(i), to obtain nGrayCorrectRightNew(i); (7) The brightness of each line on the right edge of the left image is nonlinearly smoothed according to a window of a certain width nWinWidthLeft; the brightness of each line on the left edge of the right image is nonlinearly smoothed according to a window of a certain width nWinWidthRight; (8) The left and right images are pixel-stitched to complete the stitching of two adjacent frames of images; sequential image stitching is performed in sequence to generate a panoramic image.

2. The infrared image non-overlapping stitching method according to claim 1, characterized in that: The value of nNum in step (3) is 5.

3. The infrared image non-overlapping stitching method according to claim 1, characterized in that: In step (6), the neighborhood averaging method is used for smoothing.

4. The infrared image non-overlapping stitching method according to claim 1, characterized in that: In step (7), nWinWidthLeft=100pixel, nWinWidthRight=100pixel, and a quadratic curve is used for nonlinear brightness smoothing.

5. A device for non-overlapping splicing of infrared images, characterized in that: The system consists of three parts: image acquisition module, image processing module and image display module. The image acquisition module includes a horizontal scanning module, a fast response compensation module, an optical lens module and a camera. The image processing module includes an image cropping module and an image stitching module. The image display module includes a panoramic image display module. The workflow of the stitching device includes the following steps: (1) Image acquisition: Acquisition of a circular scanning photoelectric image sequence after fast response compensation; (2) Image cropping: Through the initial image stitching, determine how many columns of pixels to crop to meet the stitching panorama requirements; (3) Image stitching: According to the method of claim 1, the brightness of adjacent frames without overlap is statistically analyzed, adjusted, and then stitched; (4) Image display: Display the stitched panoramic image.

Images