Adaptive panoramic image enhancement method and device
By using bilateral filtering and Gamma correction technology in panoramic images, the local correction mask image and correction index are calculated, and the problems of inconsistent light and darkness and insufficient adaptability of panoramic images are solved, and the balance enhancement of panoramic images and the highlight suppression of darkness are achieved. Light effect.
Patent Information
- Application Number
- CN202111030310.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-03
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-09-03
AI Technical Summary
The existing panoramic image enhancement algorithms are inconsistent in the light and dark at the edge stitching, and cannot adaptively adjust the degree of enhancement, resulting in the problem of inconsistent in the light and darkness during playback of panoramic images.
By converting panoramic images into brightness images, using bilateral filtering and Gamma correction technology, local correction mask images and Gamma correction index are calculated to achieve adaptive enhancement of panoramic images, ensuring balanced and complete edge stitching, while suppressing highlights and enhancing dark light. .
It realizes balance and completeness of edge stitching of panoramic images, suppresses highlights and enhances dark light, solves the problem of inconsistency in light and darkness of panoramic images during playback, and adjusts the degree of enhancement according to the scene.
Smart Images

Figure CN114037615B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a digital image processing technology, and in particular to an adaptive panoramic image enhancement method and device. Background Art
[0002] Panoramic images are 360° full-space images captured by professional panoramic cameras. Due to the full-recording characteristics, panoramic images have received more and more attention and applications in real estate, automobiles, cultural tourism and other fields. However, it is precisely because of the "full" characteristics that when panoramic devices capture spatial images, there will be both highlight areas and dark areas in the same panoramic image. Considering that once highlight overflow occurs, the information cannot be restored, when shooting panoramic images, the exposure parameters are usually lowered to shoot the image, and the dark information is restored later through the dark enhancement algorithm.
[0003] Image enhancement algorithms are divided into two categories, namely global enhancement and local enhancement algorithms. Since the global enhancement algorithm will enhance both highlight and dark areas at the same time, while obtaining the details of the dark area, it will expand the value and range of the highlight, resulting in the loss of highlight details, so it is not applicable to panoramic images. The local enhancement algorithm well meets the purpose of reducing the brightness of the highlight area and brightening the dark area. The local enhancement algorithm is based on the neighborhood information of the pixel point. The same brightness value can obtain different output values, and the relative brightness relationship of the output values remains unchanged.
[0004] The main problems of the prior art are as follows:
[0005] (1) In the case of panoramic images, the problem of brightness consistency at the edge joints of the panoramic image;
[0006] For the pixel-based gamma enhancement method in the local enhancement method, it evaluates the correction value of the current pixel value based on the neighborhood information, while the edge calculation window design of ordinary images will cause errors in the calculation of the edge area of the panoramic image, resulting in inconsistent brightness when the panoramic image is played in a panoramic player.
[0007] (2) The scene cannot adapt to the problem:
[0008] In the existing pixel-based Gamma enhancement method, the Gamma exponent is generally a fixed value, and the enhancement degree cannot be adaptively adjusted according to the scene conditions.
[0009] Therefore, it is urgent to solve the above problems. Summary of the invention
[0010] Purpose of the invention: The first purpose of the present invention is to provide an adaptive panoramic image enhancement method, which can achieve balanced and complete panoramic edge stitching; at the same time, through bilateral filtering and gamma correction, the effects of highlight suppression and dark light enhancement in panoramic images can be achieved.
[0011] The second object of the present invention is to provide an adaptive panoramic image enhancement device, which can achieve balanced and complete panoramic edge stitching; at the same time, through bilateral filtering and Gamma correction, the effects of highlight suppression and dark light enhancement in the panoramic image are achieved.
[0012] Technical solution: To achieve the above objectives, the present invention discloses an adaptive panoramic image enhancement method, comprising the following steps:
[0013] (1) Obtaining a brightness image of an input panoramic image, inputting a panoramic image represented by an RGB color space, converting the panoramic image from the RGB space to a space in which image brightness can be obtained, and obtaining a brightness image;
[0014] (2) Calculating a local correction mask image based on the brightness image; Calculating the local correction mask image using a bilateral filtering method;
[0015] (3) Calculate the gamma correction index based on the pixel value probability statistics of the brightness image and the local correction mask image;
[0016] (4) Calculate the enhanced panoramic image based on the gamma correction index.
[0017] The specific step of obtaining the brightness image in step (1) includes: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, where the brightness image is denoted as L, and the conversion formula is as follows:
[0018] L=0.299×R+0.587×G+0.114×B;
[0019] Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L.
[0020] Preferably, the specific method of calculating the local correction mask image in step (2) comprises the following steps:
[0021] (2.1) Expand the brightness image boundary:
[0022] Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as Lenlarge, whose width and height are W+2×K and H+2×K respectively.
[0023]
[0024] Where (i, j) represents the pixel coordinates; fill the top and bottom areas of Lenlarge with a height of K with 0;
[0025] (2.2) Perform the inversion operation on the expanded brightness image:
[0026] Lenlarge_=255-Lenlarge
[0027] (2.3) Use the bilateral filter kernel to perform bilateral filtering on Lenlarge_:
[0028]
[0029]
[0030]
[0031] Where C is the normalization coefficient, (r, c) represents the pixel coordinates in the filter window, σ1 is the standard deviation of the Gaussian function in the spatial domain, and σ2 is the standard deviation of the Gaussian function in the brightness domain;
[0032] (2.4) Mask image calculation
[0033]
[0034] Where Lmask is the calculated local mask image.
[0035] Furthermore, the specific method for calculating the gamma correction index in step (3) includes the following steps:
[0036] Calculate the exponent of Gamma correction, the exponent of Gamma correction is γ;
[0037]
[0038] In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120-140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image.
[0039]
[0040] In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value is, the greater the degree of image enhancement is.
[0041] Furthermore, the specific formula for calculating the enhanced panoramic image in step (4) is:
[0042]
[0043] The present invention discloses an adaptive panoramic image enhancement device, comprising a brightness image acquisition module, which is used to acquire a panoramic image represented by an RGB color space, convert the panoramic image from the RGB space to a space where image brightness can be obtained, and obtain and output a brightness image;
[0044] The mask image calculation module is used to receive the brightness image, calculate the local correction mask image based on the brightness image by using the bilateral filtering method, and output it;
[0045] A correction index calculation module is used to receive a local correction mask image and a brightness image, and calculate and output a gamma correction index based on pixel value probability statistics of the brightness image and the local correction mask image;
[0046] The enhanced image calculation module is used to receive the Gamma correction index and calculate the enhanced panoramic image according to the correction index.
[0047] Preferably, the specific steps of obtaining the brightness image by the brightness image acquisition module include: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, where the brightness image is denoted as L, and the conversion formula is as follows:
[0048] L=0.299×R+0.587×G+0.114×B;
[0049] Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L.
[0050] Furthermore, the specific method for calculating the local correction mask image in the mask image calculation module includes the following steps:
[0051] (2.1) Expand the brightness image boundary:
[0052] Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as Lenlarge, whose width and height are W+2×K and H+2×K respectively.
[0053]
[0054] Where (i, j) represents the pixel coordinates; fill the top and bottom areas of Lenlarge with a height of K with 0;
[0055] (2.2) Perform the inversion operation on the expanded brightness image:
[0056] Lenlarge_=255-Lenlarge
[0057] (2.3) Use the bilateral filter kernel to perform bilateral filtering on Lenlarge_:
[0058]
[0059]
[0060]
[0061] Where C is the normalization coefficient, (r, c) represents the pixel coordinates in the filter window, σ1 is the standard deviation of the Gaussian function in the spatial domain, and σ2 is the standard deviation of the Gaussian function in the brightness domain;
[0062] (2.4) Mask image calculation
[0063]
[0064] Where Lmask is the calculated local mask image.
[0065] Furthermore, the specific method for calculating the gamma correction index in the correction index calculation module includes the following steps:
[0066] Calculate the exponent of Gamma correction, the exponent of Gamma correction is γ;
[0067]
[0068] In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120-140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image.
[0069]
[0070] In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value is, the greater the degree of image enhancement is.
[0071] Preferably, the specific formula for calculating the enhanced panoramic image in the enhanced image calculation module is:
[0072]
[0073] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention utilizes the characteristics of panoramic images to obtain a brightness image with an expanded boundary, then performs bilateral filtering on the expanded image, extracts a mask image, calculates the Gamma correction index based on the probability statistics of the pixel values of the brightness image and the local correction mask image, and finally calculates the enhanced panoramic image according to the correction index; thereby achieving balanced and complete panoramic edge stitching; at the same time, bilateral filtering and Gamma correction are used to achieve the effects of highlight suppression and dark light enhancement in the panoramic image. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 It is a schematic diagram of the process of the present invention;
[0075] Figure 2 A schematic diagram of a process for calculating a local correction mask image in the present invention;
[0076] Figure 3 is the original panoramic image in the embodiment of the present invention;
[0077] Figure 4 This is the enhanced panoramic image in the embodiment of the present invention. DETAILED DESCRIPTION
[0078] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0079] like Figure 1 and Figure 2 As shown, an adaptive panoramic image enhancement method comprises the following steps:
[0080] (1) Obtaining a brightness image of an input panoramic image, inputting a panoramic image represented by an RGB color space, converting the panoramic image from the RGB space to a space in which image brightness can be obtained, and obtaining a brightness image;
[0081] The specific steps of obtaining the brightness image include: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, where the brightness image is denoted as L, and the conversion formula is as follows:
[0082] L=0.299×R+0.587×G+0.114×B;
[0083] Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L;
[0084] (2) Calculating a local correction mask image based on the brightness image; Calculating the local correction mask image using a bilateral filtering method;
[0085] The specific method for calculating the local correction mask image includes the following steps:
[0086] (2.1) Expand the brightness image boundary:
[0087] Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as Lenlarge, whose width and height are W+2×K and H+2×K respectively.
[0088]
[0089] Where (i, j) represents the pixel coordinates; fill the top and bottom areas of Lenlarge with a height of K with 0;
[0090] (2.2) Perform the inversion operation on the expanded brightness image:
[0091] Lenlarge_=255-Lenlarge
[0092] (2.3) Use the bilateral filter kernel to perform bilateral filtering on Lenlarge_:
[0093]
[0094]
[0095]
[0096] Where C is the normalization coefficient, (r, c) represents the pixel coordinates in the filter window, σ1 is the standard deviation of the Gaussian function in the spatial domain, and σ2 is the standard deviation of the Gaussian function in the brightness domain;
[0097] (2.4) Mask image calculation
[0098]
[0099] Where Lmask is the calculated local mask image;
[0100] (3) Calculate the gamma correction index based on the pixel value probability statistics of the brightness image and the local correction mask image;
[0101] The specific method for calculating the gamma correction index includes the following steps:
[0102] Calculate the exponent of Gamma correction, the exponent of Gamma correction is γ;
[0103]
[0104] In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120-140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image.
[0105]
[0106] In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value, the greater the degree of image enhancement;
[0107] (4) Calculate the enhanced panoramic image according to the correction index, where the specific formula for calculating the enhanced panoramic image is:
[0108]
[0109] The present invention discloses an adaptive panoramic image enhancement device, comprising a brightness image acquisition module, which is used to acquire a panoramic image represented by an RGB color space, convert the panoramic image from the RGB space to a space where the image brightness can be obtained, obtain the brightness image and output it; the specific steps of the brightness image acquisition module to obtain the brightness image include: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, recording the brightness image as L, and the conversion formula is as follows:
[0110] L=0.299×R+0.587×G+0.114×B;
[0111] Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L;
[0112] The mask image calculation module is used to receive the brightness image, calculate the local correction mask image based on the brightness image using the bilateral filtering method, and output it; the specific method of calculating the local correction mask image in the mask image calculation module includes the following steps:
[0113] (2.1) Expand the brightness image boundary:
[0114] Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as Lenlarge, whose width and height are W+2×K and H+2×K respectively.
[0115]
[0116] Where (i, j) represents the pixel coordinates; fill the top and bottom areas of Lenlarge with a height of K with 0;
[0117] (2.2) Perform the inversion operation on the expanded brightness image:
[0118] Lenlarge_=255-Lenlarge
[0119] (2.3) Use the bilateral filter kernel to perform bilateral filtering on Lenlarge_:
[0120]
[0121]
[0122]
[0123] Where C is the normalization coefficient, (r, c) represents the pixel coordinates in the filter window, σ1 is the standard deviation of the Gaussian function in the spatial domain, and σ2 is the standard deviation of the Gaussian function in the brightness domain;
[0124] (2.4) Mask image calculation
[0125]
[0126] Where Lmask is the calculated local mask image;
[0127] The correction index calculation module is used to receive the local correction mask image and the brightness image, and calculate the gamma correction index based on the pixel value probability statistics of the brightness image and the local correction mask image and output it; the specific method of calculating the gamma correction index in the correction index calculation module includes the following steps:
[0128] Calculate the exponent of Gamma correction, the exponent of Gamma correction is γ;
[0129]
[0130] In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120-140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image.
[0131]
[0132] In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value, the greater the degree of image enhancement;
[0133] The enhanced image calculation module is used to receive the Gamma correction index and calculate the enhanced panoramic image according to the correction index; the specific formula for calculating the enhanced panoramic image in the enhanced image calculation module is:
[0134]
[0135] Figure 3 is the original panoramic image, and after being processed by the adaptive panoramic image enhancement method and device of the present invention, an enhanced panoramic image can be seen, such as Figure 4 As shown, the effects of highlight suppression and dark light enhancement in the panoramic image are achieved simultaneously.
Claims
1. An adaptive panoramic image enhancement method, characterized in that: The steps include: (1) Obtaining a brightness image of an input panoramic image, inputting a panoramic image represented by an RGB color space, converting the panoramic image from the RGB space to a space where image brightness can be obtained, and obtaining a brightness image; The specific steps of obtaining the brightness image in step (1) include: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, where the brightness image is denoted as L, and the conversion formula is as follows: ; Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L; (2) Calculate the local correction mask image based on the brightness image; Calculate the local correction mask image using the bilateral filtering method; The specific method for calculating the local correction mask image in step (2) comprises the following steps: (2.1) Expand the brightness image boundary: Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as L enlarge , whose width and height are W+2×K and H+2×K respectively, then , where (i, j) represents the pixel coordinates; L enlarge The top and bottom areas with a height of K are filled with 0; (2.2) Perform the inversion operation on the expanded brightness image: , (2.3) Use the bilateral filtering kernel to perform bilateral filtering on Lenlarge_: , , , Where C is the normalization coefficient, (r, c) represents the pixel coordinates within the filter window, is the standard deviation of the spatial Gaussian function, is the standard deviation of the Gaussian function in the brightness domain; (2.4) Mask image calculation , In the formula, That is the calculated local mask image; (3) Calculate the gamma correction index based on the pixel value probability statistics of the brightness image and the local correction mask image; In step (3), the calculation The specific method of correcting the index includes the following steps: calculate Corrected index, The exponent of the correction is γ; , In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120~140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image. , In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value, the greater the degree of image enhancement; (4) Calculate the enhanced panoramic image according to the gamma correction index; The specific formula for calculating the enhanced panoramic image in step (4) is: 。 2. An adaptive panoramic image enhancement device, characterized in that: It includes a brightness image acquisition module, which is used to acquire a panoramic image represented by an RGB color space, convert the panoramic image from the RGB space to a space where image brightness can be obtained, and obtain and output a brightness image; The specific steps of the brightness image acquisition module to obtain the brightness image include: converting the panoramic image from the RGB space to the YUV space to obtain the brightness image, where the brightness image is denoted as L, and the conversion formula is as follows: ; Alternatively, the panoramic image is converted from the RGB space to the HSV space to obtain a brightness image, and the brightness image is recorded as L; The mask image calculation module is used to receive the brightness image, calculate the local correction mask image based on the brightness image by using the bilateral filtering method, and output it; The specific method for calculating the local correction mask image in the mask image calculation module comprises the following steps: (2.1) Expand the brightness image boundary: Set the radius of the bilateral filter window to K, the width and height of the brightness image to W and H respectively, and the expanded brightness image is recorded as L enlarge , whose width and height are W+2×K and H+2×K respectively, then , where (i, j) represents the pixel coordinates; L enlarge The top and bottom areas with a height of K are filled with 0; (2.2) Perform the inversion operation on the expanded brightness image: , (2.3) Use the bilateral filtering kernel to perform bilateral filtering on Lenlarge_: , , , Where C is the normalization coefficient, (r, c) represents the pixel coordinates within the filter window, is the standard deviation of the spatial Gaussian function, is the standard deviation of the Gaussian function in the brightness domain; (2.4) Mask image calculation: , In the formula, That is the calculated local mask image; The correction index calculation module is used to receive the local correction mask image and the brightness image, and calculate the local correction mask image based on the pixel value probability statistics of the brightness image Corrected index and output; The correction index calculation module calculates The specific method of correcting the index includes the following steps: calculate Corrected index, The exponent of the correction is γ; , In the formula, the size of pa affects the overall brightness of the enhanced image, and the value range of pa is 120~140. The size of α affects the degree of enhancement, and α is calculated by calculating the probability statistics of the brightness image. , In the formula, Imean represents the mean of the brightness image, p represents the probability of each pixel value appearing, and the larger the α value, the greater the degree of image enhancement; An enhanced image calculation module, used for receiving a Gamma correction index and calculating an enhanced panoramic image according to the correction index; The specific formula for calculating the enhanced panoramic image in the enhanced image calculation module is: 。
