A multi-aperture optical panoramic image view enhancement method

Abstract

The application discloses a kind of multi-aperture optical panoramic image view enhancement method, for under the condition of limited optical field of view, the purpose of realizing large scene target monitoring by multi-aperture optical array image splicing technology, specifically includes the following steps: 1) obtaining multi-aperture image;2) format conversion is carried out to multi-aperture image;3) statistics single-aperture image histogram and multi-aperture image histogram;4) the calibration base value of this aperture is calculated by single-aperture image histogram;5) the segmentation threshold of highlight and low light area is calculated by multi-aperture image histogram;6) the calibration parameter of panoramic image is calculated;7) real-time calibration is carried out to panoramic image using calibration parameter.

Description

Technical Field

[0001] This invention relates to a method for enhancing visual perception, and more particularly to a method for enhancing the visual perception of multi-aperture optical panoramic images. Background Technology

[0002] Traditional optical imaging systems suffer from a narrow field of view, a large field of view, and high resolution, which are becoming stumbling blocks to the rapid development of optical imaging systems. In order to effectively solve the contradiction between the constraints of optical field of view and spatial resolution, panoramic images generated by multi-aperture imaging and image stitching technology can see the details of the target while ensuring wide-area monitoring, which has attracted widespread attention and research from scholars in the field. Ming-Shing Su et al. used wavelets for multi-scale fusion. However, this method only performs local fusion processing in the overlapping area, and the global image cannot be guaranteed. Anat Levin et al. adopted the gradient domain fusion method for image stitching, which reduced the influence of the color difference of the original image on the fusion result. However, for images with obvious color and brightness differences, the generated panoramic image will be distorted. Existing panoramic image generation methods have the following disadvantages: (1) For images with obvious color and brightness differences, the distortion of the stitched panoramic image is large; (2) In general, it cannot be guaranteed that there is no obvious stitching between adjacent images; (3) Most panoramic image generation algorithms are computationally complex and consume a lot of hardware resources. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for enhancing the visual view of multi-aperture optical panoramic images, which addresses the shortcomings of the existing technology.

[0004] To address the aforementioned technical problems, this invention discloses a method for enhancing the visual perspective of multi-aperture optical panoramic images, comprising the following steps:

[0005] Step 1: Acquire multi-aperture images;

[0006] Step 2: Convert the format of the multi-aperture images to unify all images into grayscale image format;

[0007] Step 3: Calculate the histograms of the single-aperture and multi-aperture images above;

[0008] Step 4: Calculate the calibration baseline value for each single aperture image using the single aperture image histogram;

[0009] Step 5: Calculate the segmentation thresholds for highlight and low-brightness regions using the histogram of the multi-aperture image;

[0010] Step 6: Calculate the calibration parameters of the panoramic image based on the calibration baseline and segmentation threshold of each aperture; the panoramic image is synthesized from multi-aperture images;

[0011] Step 7: Perform real-time calibration of the panoramic image using calibration parameters to complete the visual enhancement of the multi-aperture optical panoramic image.

[0012] Furthermore, the multi-aperture image mentioned in step 1 refers to the number of visible light cameras or infrared thermal imagers. Four channels, acquiring optical images; the image types include at least: visible light or infrared images.

[0013] Furthermore, the format conversion described in step 2 includes:

[0014] Convert the format of the multi-aperture image: if the original multi-aperture image is in RGB format, convert it to YUV format; if the original multi-aperture image is in grayscale, leave it unchanged.

[0015] Furthermore, the histogram mentioned in step 3, i.e., the horizontal axis represents the brightness value. The vertical axis represents the brightness value. The probability of occurrence.

[0016] Furthermore, the histogram of the single-aperture image mentioned in step 3 includes:

[0017] Assuming a single aperture image is , ..., , It is the first Image of a single aperture medium pixel brightness value, It is an image Brightness value If the probability of a pixel appearing is:

[0018]

[0019] In the formula, This indicates the number of single-aperture images in a multi-aperture image. It's the pixel position. It is an image Brightness value The number of pixels, It is an image The total number of pixels.

[0020] Furthermore, the histogram of the multi-aperture image described in step 3 includes:

[0021] Assuming the multi-aperture image is , Multi-aperture image medium pixel brightness value, It is an image Brightness value The probability of a pixel appearing, then

[0022]

[0023] In the formula, It's the pixel position. The brightness value is The number of pixels, It represents the total number of pixels.

[0024] Furthermore, the calculation of the calibration baseline value of each single aperture image using the single aperture image histogram in step 4 specifically includes:

[0025] Step 4-1, Set the image histogram The high and low ratio coefficients are the same, and the high and low ratio coefficients are preset values. Image of a single aperture Medium brightness value The range of values ​​is ;

[0026] Step 4-2, Calculate the high threshold and low threshold The details are as follows:

[0027] The high threshold The calculation method is as follows:

[0028] When the brightness value is within the range The number of pixels is equal to the preset value multiplied by the image. Total number of pixels At this time, the brightness value It is a high threshold ;

[0029] The low threshold The calculation method is as follows:

[0030] When the brightness value is within the range The number of pixels is equal to the preset value multiplied by the image. Total number of pixels At this time, the brightness value It is a low threshold ;

[0031] Step 4-3: Calculate the calibration baseline value for each single aperture image. Specific methods include:

[0032] image calibration baseline The brightness value is greater than and less than The average pixel value.

[0033] Furthermore, the calculation of the segmentation thresholds for the highlight and low-brightness regions described in step 5 specifically includes:

[0034] Step 5-1, Set the multi-aperture image histogram The high / low ratio coefficient is the same as the high / low ratio coefficient set in step 4-1;

[0035] Step 5-2: Using the method in Step 4-2, calculate the high threshold. and low threshold ;

[0036] Step 5-3: Calculate the multi-aperture image based on threshold segmentation theory. Medium brightness value greater than and less than Segmentation thresholds for highlight and low brightness regions .

[0037] Furthermore, the calculation of calibration parameters for the panoramic image in step 6 specifically includes:

[0038] In a panoramic image, the calibration baseline value of the highlighted area The brightness value is greater than and less than The pixel mean; the calibration baseline for low-brightness areas. The brightness value is greater than and less than The average pixel value;

[0039] Calibration parameters for panoramic images Represented as:

[0040]

[0041] In the formula, Indicates the first Aperture.

[0042] Furthermore, the real-time calibration of the panoramic image using calibration parameters described in step 7 specifically includes:

[0043] No. Single aperture image calibration image Represented as:

[0044]

[0045] After calibrating all the individual aperture images using the above formula, the calibrated panoramic image is output through image stitching.

[0046] Beneficial effects:

[0047] (1) For images with obvious color and brightness differences, the panoramic images generated by stitching have less distortion, richer and more significant image details, which is more beneficial for users to monitor targets;

[0048] (2) The color transition between adjacent images is smooth, with no obvious splicing gaps, resulting in good visual appeal;

[0049] (3) The panoramic image visual enhancement method proposed in this invention does not involve high-order operations, has a simple algorithm structure, and is beneficial for real-time monitoring of high-speed targets. Attached Figure Description

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0051] Figure 1 This is a schematic diagram of the workflow of the present invention.

[0052] Figure 2 A schematic diagram of the image stitching effect in one embodiment of the present invention. Detailed Implementation

[0053] This invention provides a method for enhancing the view of multi-aperture optical panoramic images. It is used to fuse and stitch images from multiple optical sensors and enhance the images, providing wide-scene panoramic images to meet the needs of users monitoring large-scene targets. The method mainly includes: acquiring multi-aperture images (images output by multiple optical lenses); converting the format of the multi-aperture images; calculating the histograms of single-aperture and multi-aperture images (single-aperture images refer to images output by a single optical lens); calculating the calibration baseline value of the aperture using the single-aperture image histogram; calculating the segmentation thresholds for high-brightness and low-brightness regions using the multi-aperture image histogram; calculating the calibration parameters of the panoramic image; and performing real-time calibration of the panoramic image using the calibration parameters, as detailed below:

[0054] Step 1: Acquire multi-aperture images;

[0055] Multi-aperture imaging refers to the number of visible light cameras or infrared thermal imagers. Four channels; image types include visible light, infrared images, etc.

[0056] Visible light images refer to high-definition (2K) or ultra-high-definition (4K) RGB color images; infrared images refer to... , or A grayscale image.

[0057] Step 2: Convert the format of the multi-aperture image;

[0058] Convert the format of multi-aperture images: if the original image type is RGB, convert it to YUV format; if the original image type is grayscale, leave it unchanged.

[0059] The formula for converting a color image from RGB to YUV format is as follows:

[0060] Y = 0.2990R + 0.5870G + 0.1440B

[0061] U = -0.1684R - 0.3316G + 0.5B + ​​128

[0062] V = 0.5R - 0.4187G - 0.0813B + 128

[0063] Step 3: Calculate the histograms of single-aperture and multi-aperture images;

[0064] Assuming a single aperture image is , ..., , It is an image Pixels brightness value, It is an image Brightness value The probability of occurrence, then

[0065]

[0066] In the formula, Indicates the number of multi-aperture images. It's the pixel position. It is an image Brightness value The number of pixels, It is an image The total number of pixels.

[0067] Assuming the multi-aperture image is , It is an image Pixels brightness value, It is an image Brightness value The probability of occurrence, then

[0068]

[0069] In the formula, It's the pixel position. The brightness value is The number of pixels, It represents the total number of pixels.

[0070] Step 4: Calculate the calibration baseline value for the aperture using the histogram of the single aperture image;

[0071] For the Aperture, setting image Histogram The high and low ratio coefficients are set to be the same for each aperture, and the high threshold is calculated. and low threshold Then the image calibration baseline The brightness value is greater than and less than The average pixel value.

[0072] Step 5: Calculate the segmentation thresholds for highlight and low-brightness regions using the histogram of the multi-aperture image;

[0073] Setting up multi-aperture images histogram The high and low ratio coefficients are the same as those in step 4, and the high threshold is calculated. and low threshold Based on Otsu's adaptive threshold segmentation theory, calculate Medium brightness value greater than and less than Segmentation thresholds for highlight and low-brightness regions .

[0074] Step 6: Calculate the calibration parameters for the panoramic image;

[0075] Calibration base value of the highlighted area of ​​the panoramic image The brightness value is greater than and less than The pixel mean; the calibration baseline for low-brightness areas. The brightness value is greater than and less than The average pixel value.

[0076] Calibration parameters for panoramic images Represented as:

[0077]

[0078] In the formula, Indicates the first Aperture.

[0079] Step 7: Perform real-time calibration of the panoramic image using calibration parameters.

[0080] No. Aperture Image Calibrate images Represented as:

[0081]

[0082] From calibration image A panoramic image is output after image stitching.

[0083] Example:

[0084] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0085] like Figure 1 As shown, this invention discloses a method for enhancing the visual view of multi-aperture optical panoramic images, comprising the following steps:

[0086] (1) Acquire multi-aperture images;

[0087] Here, take bus route 8 and bus route 19. Taking a color image (RGB format) as an example, the visual enhancement method of the present invention will be described in detail.

[0088] (2) Convert the format of the multi-aperture image;

[0089] The formula for converting a color image from RGB to YUV format is as follows:

[0090] Y = 0.2990R + 0.5870G + 0.1440B

[0091] U = -0.1684R - 0.3316G + 0.5B + ​​128

[0092] V = 0.5R - 0.4187G - 0.0813B + 128

[0093] (3) Statistical histograms of single-aperture and multi-aperture images;

[0094] According to the probability model Statistical analysis of images with 8 apertures , ..., The histogram.

[0095] Image with the first aperture For example, ,image Medium brightness value The probability of occurrence is the image Medium brightness value quantity Divide by image Total number of all brightness values ,in, The range of values ​​for is .

[0096] According to the probability model Statistical histogram of multi-aperture images (4) Calculate the calibration baseline value of the aperture using the histogram of the single aperture image;

[0097] Eight high and low proportional coefficients are set for different apertures. For the first... Aperture, setting image Histogram Both the high and low proportionality coefficients are 0.05. Taking the first aperture as an example, the high threshold... The calculation result is 223, which is the low threshold. The calculation result is 76; then the image calibration baseline It is the average value of pixels with a brightness value greater than 76 and less than 223, and the calculated result is 159.

[0098] (5) Calculate the segmentation thresholds for bright and dark regions using the histogram of the multi-aperture image;

[0099] Setting up multi-aperture images is Histogram The high and low ratio coefficients are set to 0.05, the same as those in step 4. The high threshold of 241 and the low threshold of 35 are calculated. Based on Otsu's threshold segmentation theory, the following calculations are performed. The threshold for segmenting bright and low brightness areas is 178, with a brightness value greater than 35 and less than 241.

[0100] (6) Calculate the calibration parameters of the panoramic image;

[0101] Calibration base value of the highlighted area of ​​the panoramic image It is the average pixel value for low brightness values ​​greater than 35 and less than 178, with a calculated result of 142; the calibration base value for the high brightness area. It is the average value of pixels with a brightness value greater than 178 and less than 241, and the calculated result is 213.

[0102] Calibration parameters for panoramic images Represented as:

[0103]

[0104] In the formula, Indicates the first Aperture.

[0105] (7) Perform real-time calibration of panoramic images using calibration parameters.

[0106] For 8 apertures, the first Aperture Image Calibrate images Represented as:

[0107]

[0108] From calibration image The panoramic image is output after image stitching, such as Figure 2 As shown.

[0109] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a multi-aperture optical panoramic image enhancement method, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0110] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0111] This invention provides a concept and method for enhancing the visual perspective of multi-aperture optical panoramic images. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for enhancing the visual perspective of multi-aperture optical panoramic images, characterized in that, Includes the following steps: Step 1: Acquire multi-aperture images; Step 2: Convert the format of the multi-aperture images to unify all images into grayscale image format; Step 3: Calculate the histograms of each single-aperture image and the multi-aperture image in the multi-aperture image; Step 4: Calculate the calibration baseline value for each single aperture image using the single aperture image histogram; Step 5: Calculate the segmentation thresholds for highlight and low-brightness regions using the multi-aperture image histogram. and ; Step 6: Calculate the calibration parameters of the panoramic image based on the calibration baseline and segmentation threshold of each aperture; the panoramic image is synthesized from multi-aperture images; Step 7: Perform real-time calibration of the panoramic image using calibration parameters to complete the visual enhancement of the multi-aperture optical panoramic image; Specifically, step 6, which involves calculating the calibration parameters for the panoramic image, includes: In a panoramic image, the calibration baseline value of the highlighted area The brightness value is greater than and less than The pixel mean; the calibration baseline for low-brightness areas. The brightness value is greater than and less than The average pixel value; Calibration parameters for panoramic images Represented as: ； In the formula, Indicates the first One aperture, For calibration baseline value, This is the threshold for segmenting the bright and dark regions.

2. The method for enhancing the visual view of a multi-aperture optical panoramic image according to claim 1, characterized in that, The multi-aperture image mentioned in step 1 refers to the number of visible light cameras or infrared thermal imagers. 4 channels, the acquired optical images; Image types must include at least: visible light or infrared images.

3. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 2, characterized in that, The format conversion described in step 2 includes: Convert the format of the multi-aperture image: if the original multi-aperture image is in RGB format, convert it to YUV format; if the original multi-aperture image is in grayscale, leave it unchanged.

4. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 3, characterized in that, The histogram mentioned in step 3, i.e., the horizontal axis represents the brightness value. The vertical axis represents the brightness value. The probability of occurrence.

5. The method for enhancing the visual view of a multi-aperture optical panoramic image according to claim 4, characterized in that, The histogram of the single aperture image mentioned in step 3 includes: Assuming a single aperture image is , ..., , It is the first Image of a single aperture medium pixel brightness value, It is an image Brightness value If the probability of a pixel appearing is: ； In the formula, This indicates the number of single-aperture images in a multi-aperture image. It's the pixel position. It is an image Brightness value The number of pixels, It is an image The total number of pixels.

6. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 5, characterized in that, The histogram of the multi-aperture image mentioned in step 3 includes: Assuming the multi-aperture image is , Multi-aperture image medium pixel brightness value, It is an image Brightness value The probability of a pixel appearing, then ； In the formula, It's the pixel position. The brightness value is The number of pixels, It represents the total number of pixels.

7. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 6, characterized in that, Step 4, which involves calculating the calibration baseline value of each single aperture image using the single aperture image histogram, specifically includes: Step 4-1, Set the image histogram The high and low ratio coefficients are the same, and the high and low ratio coefficients are preset values. Image of a single aperture Medium brightness value The range of values ​​is ; Step 4-2, Calculate the high threshold and low threshold The details are as follows: The high threshold The calculation method is as follows: When the brightness value is within the range The number of pixels is equal to the preset value multiplied by the image. Total number of pixels At this time, the brightness value It is a high threshold ; The low threshold The calculation method is as follows: When the brightness value is within the range The number of pixels is equal to the preset value multiplied by the image. Total number of pixels At this time, the brightness value It is a low threshold ; Step 4-3: Calculate the calibration baseline value for each single aperture image. Specific methods include: image calibration baseline The brightness value is greater than and less than The average pixel value.

8. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 7, characterized in that, Step 5, which involves calculating the segmentation thresholds for highlight and low-brightness regions, specifically includes: Step 5-1, Set the multi-aperture image histogram The high / low ratio coefficient is the same as the high / low ratio coefficient set in step 4-1; Step 5-2: Using the method in Step 4-2, calculate the high threshold. and low threshold ; Step 5-3: Calculate the multi-aperture image based on threshold segmentation theory. Medium brightness value greater than and less than Segmentation thresholds for highlight and low brightness regions .

9. The method for enhancing the visual perspective of a multi-aperture optical panoramic image according to claim 8, characterized in that, Step 7, which describes real-time calibration of the panoramic image using calibration parameters, specifically includes: No. Single aperture image calibration image Represented as: ； After calibrating all individual aperture images using the above formula, the calibrated panoramic image is output through image stitching.

Images