An efficient multi-exposure image block-based fusion method

Abstract

The application discloses a high-efficiency multi-exposure image block fusion method, and aims at the problems of large calculation amount and high calculation complexity caused by too many calculation times of two-dimensional Gaussian function values in a multi-exposure image block fusion algorithm, a method for deducing periodicity of two-dimensional Gaussian function values required by pixel fusion by using pixel position geometric symmetry is used to find equivalent relations among two-dimensional Gaussian function values required by each pixel fusion, so that the minimum calculation times of the two-dimensional Gaussian function values are obtained, the calculation amount and the calculation complexity of the algorithm can be effectively reduced by means of multiple reuse of one calculation, the calculation speed is improved, and the area required by deployment of the algorithm on a programmable logic device can be effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a high-efficiency multi-exposure image block fusion method. BACKGROUND

[0002] The dynamic range of an image can be defined as the ratio of the maximum brightness to the minimum brightness that can be represented in the image. The greater the dynamic range of an image, the greater the range of brightness that can be displayed, and the more effective information that can be represented. High dynamic range imaging (HDR) technology can obtain an image with a greater dynamic range. An HDR image often has a higher contrast and richer image detail information. This technology is widely used in digital photography, medical imaging and other fields.

[0003] At present, there are mainly two ways to obtain an HDR image: one is to directly generate an HDR image using a professional hardware imaging device, but the related equipment is expensive and difficult to popularize, and it is still difficult to achieve the true dynamic range of a natural scene; the other is to generate an HDR image using one or more low dynamic range images through tone mapping technology, multi-exposure image fusion technology and other methods.

[0004] Among them, the multi-exposure image block fusion algorithm in the multi-exposure image fusion technology can generate a better HDR image. This algorithm belongs to the fusion method in the image spatial domain, and introduces image brightness smoothing processing to remove the obvious boundaries caused by block fusion. Compared with the common Laplacian pyramid decomposition type multi-exposure fusion algorithm, the multi-exposure image block fusion algorithm uses fewer data sources, the operation is relatively simple, the structure is more optimal, and certain hardware resources can be saved, which has more advantages in application.

[0005] However, the multi-exposure image block fusion algorithm still has defects, mainly including the following problems:

[0006] (1) The problem of excessive calculation amount and memory access amount: in the multi-exposure image block fusion algorithm, when a pixel in an image block is fused, the gray values of the pixels corresponding to all image blocks at the same position need to be obtained, and the same number of two-dimensional Gaussian function values need to be calculated to further calculate the fusion weights of the pixels at the corresponding positions. The calculation of the two-dimensional Gaussian function value involves power and exponential operations, resulting in a large calculation amount and algorithm delay of the algorithm, which will consume a large amount of computing resources. If the two-dimensional Gaussian function values required by each pixel are calculated and stored in advance, 9 times the total number of pixels of two-dimensional Gaussian function values need to be stored, which will cause great pressure on the storage space;

[0007] (2) Problem of repeated calculation of two-dimensional Gaussian function value: Since the two-dimensional Gaussian function value required by each pixel to be fused depends on the distance between the pixel to be fused and the center of each image block, the two-dimensional Gaussian function values ​​required by pixels to be fused at different locations partially overlap. The repeated calculation of these overlapping two-dimensional Gaussian function values ​​is an unnecessary repetitive calculation, which will significantly increase the calculation delay and calculation power consumption. Summary of the Invention

[0008] To address the problems of the aforementioned multi-exposure image block fusion methods, this invention proposes an efficient multi-exposure image block fusion method. This method aims to effectively reduce the computational load and complexity of the algorithm, while improving computational speed, thereby effectively reducing the area required for the fusion method to be deployed on a programmable logic device.

[0009] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0010] The present invention provides an efficient multi-exposure image block fusion method, characterized by the following steps:

[0011] S1. Acquire Q original images with different exposure times, resolutions of S×T, and color standard of YCbCr, and divide each original image into segments with side length N. There are three square image blocks; any image block in each original image is denoted as the image block in the i-th row and j-th column, and...

[0012] S2. Statistically analyze the grayscale distribution of each image patch in the original images. Based on the grayscale distribution of each image patch in the original images, calculate the information entropy of image patches at the same location in Q original images. Then, select the image patch corresponding to the maximum information entropy to obtain... The maximum information entropy image blocks are combined into an information entropy image; the Cb and Cr channel values ​​of each pixel in the information entropy image are stored sequentially into an S×T×2 three-dimensional color channel matrix A;

[0013] S3. Calculate the fusion weight of each pixel in each image block of the information entropy image;

[0014] S3.1. Let V be any image block in the information entropy image, and let (u,v) be the coordinates of any pixel r in the image block V, where u,v∈[0,N-1].

[0015] S3.2. In the information entropy image, centering on image block V and designating it as the image block in row 0 and column 0, a region of dimension H×H is defined, where H is an odd number; any image block within the H×H region is designated as the image block in row p and column q; and

[0016] S3.3, calculating the fusion weight W of the pixel point r for the pixel point at the (u, v) position of the pth row qth column image block in the HxH image block region by using formula (1) p,q (u, v), so as to obtain the fusion weight of each pixel point of each image block in the information entropy image, and store the fusion weight in the three-dimensional fusion weight matrix U of NxNxH 2 ;

[0017]

[0018] In formula (1), G m,n (u, v) represents the function value of the two-dimensional Gaussian function centered on the center pixel point of the mth row nth column image block in the HxH image block region at the (u, v) position in the image block V, G p,q (u, v) represents the function value of the two-dimensional Gaussian function centered on the center point of the pth row qth column image block in the HxH image block region at the (u, v) position in the image block V, and has:

[0019]

[0020] In formula (2), d pq (u, v) is the distance between the pixel point at the (u, v) position in the image block V and the center point of the pth row qth column image block in the HxH image block region, and sigma is the standard deviation of the two-dimensional Gaussian function;

[0021] S4, image fusion is performed to generate a multi-exposure fusion image:

[0022] S4.1, initializing i = 1;

[0023] S4.2, initializing j = 1;

[0024] S4.3, calculating the fusion pixel point gray value O ij (u, v) at the (u, v) position in the ith row jth column image block of the multi-exposure fusion image by using formula (3), and sequentially storing in the two-dimensional fusion gray value matrix B of SxT:

[0025]

[0026] In formula (3), I i+p,j+q (u, v) is the gray value of the pixel point at the (u, v) position in the (i+p)th row (j+q)th column image block of the original image where the ith row jth column image block is located;

[0027] S4.4, after assigning j+1 to j, judging whether the following condition is established: O

[0028] S4.5, after assigning i+1 to i, judging whether is established, if yes, returning to S4.2; otherwise, executing S5;

[0029] S5, merging the two-dimensional fusion gray value matrix B of SxT and the three-dimensional color channel matrix A of SxTx2, thereby obtaining a three-dimensional multi-exposure fusion image of SxTx3 and outputting.

[0030] The electronic device of the present application comprises a memory and a processor, and the feature is that the memory is used to store a program supporting the processor to execute the multi-exposure image block fusion method, and the processor is configured to execute the program stored in the memory.

[0031] The computer readable storage medium of the present application has a computer program stored thereon, and the feature is that the computer program executes the steps of the multi-exposure image block fusion method when run by a processor.

[0032] Compared with the prior art, the beneficial effects of the present application are reflected in the following aspects:

[0033] 1. The present application proposes to use only HxH image blocks centered on the image blocks where the pixels to be fused are located for block fusion, and in this case, the fusion weights required for the block fusion of the pixels in each image block are the same, and only the fusion weight required for the block fusion of the pixels in one image block needs to be calculated, thereby effectively reducing the number of fusion weight calculations and the calculation amount of the fused pixel gray values.

[0034] 2. The present application proposes to utilize the symmetry feature of the two-dimensional Gaussian function values used for the calculation of the fusion weights of the position-symmetrical pixels in the same image block, thereby reducing the number of two-dimensional Gaussian function calculations, and the number of two-dimensional Gaussian function value calculations can be reduced by 75%. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is the flowchart of the method of the present application;

[0036] Figure 2 is the block fusion principle diagram of the method of the present application;

[0037] Figure 3 is the two-dimensional Gaussian function value symmetry diagram of the method of the present application;

[0038] Figure 4 is the four-equal-part diagram of the method of the present application for one image block. DETAILED DESCRIPTION

[0039] The present application will be further described below in combination with the drawings and examples.

[0040] In this embodiment, an efficient multi-exposure image block fusion method is to divide the input multi-exposure images into blocks, calculate the gray scale and information entropy, and select the image block with the maximum information entropy. The fusion weight required by each pixel in an image block is calculated, and then the pixels in each image block are fused to obtain a multi-exposure fused image. Specifically, as shown in Figure 1 , the method is performed in the following steps:

[0041] S1, obtaining Q original images with different exposure times, a resolution of SxT=1920x1080, and a color standard of YCbCr, and dividing each original image into 144 square image blocks with a side length of N=120; any one image block in each original image is recorded as the image block in the i-th row and the j-th column, and i∈[1,9], j∈[1,16];

[0042] S2, calculating the gray scale distribution of each image block in the original image, and calculating the information entropy of the image blocks at the same position in the Q original images according to the gray scale distribution of each image block in the original image, thereby selecting the image block corresponding to the maximum information entropy to obtain 144 image blocks with the maximum information entropy and combining them into an information entropy image; the values of Cb and Cr channels of each pixel point in the information entropy image are sequentially stored in a three-dimensional color channel matrix A with a size of SxTx2=1920x1080x2;

[0043] S3, calculating the fusion weight of each pixel in each image block in the information entropy image; the specific fusion weight calculation method is as follows:

[0044] S3.1, any one image block in the information entropy image is recorded as V, and the coordinates of any one pixel point r in the image block V are recorded as (u, v), u, v∈[0, 120];

[0045] S3.2, as shown in Figure 2 , taking the image block V as the center in the information entropy image and as the image block in the 0-th row and the 0-th column, thereby dividing a 3x3 image block region; any one image block in the 3x3 image block region is recorded as the image block in the p-th row and the q-th column; and p, q∈[-1, 1];

[0046] S3.3, the fusion weight W p,q (u, v) of the pixel point r for the pixel point at the (u, v) position of the image block in the p-th row and the q-th column in the 3x3 image block region is calculated by using formula (1), thereby storing the fusion weight of each pixel point in each image block in the information entropy image into a three-dimensional fusion weight matrix U with a size of NxNxH 2 =120x120x9;

[0047] As shown in Figure 3 ​As shown in each image block, the position-symmetrical pixels have symmetry with the distance between the center of each pixel block in the 3x3 image block region, that is, the fusion weight of the position-symmetrical pixels has symmetry. Thus, each image block is equally divided into four regions as shown Figure 4 The fusion weight of the pixels in the A1 region is calculated, and the fusion weights of the pixels in the remaining three regions can be directly obtained by symmetry. The specific calculation method is as follows:

[0048]

[0049] In formula (1), G m,n (u,v) represents the function value of the two-dimensional Gaussian function centered at the center pixel point of the image block in the mth row and the nth column in the 3x3 image block region at the (u,v) position in the image block V, G p,q (u,v) represents the function value of the two-dimensional Gaussian function centered at the center pixel point of the image block in the pth row and the qth column in the 3x3 image block region at the (u,v) position in the image block V, and has:

[0050]

[0051] In formula (2), d pq (u,v) is the distance between the pixel point at the (u,v) position in the image block V and the center pixel point of the image block in the pth row and the qth column in the HxH image block region, and σ is the standard deviation of the two-dimensional Gaussian function, and σ=50 is taken;

[0052] If the traditional multi-exposure image block fusion method is used, 1920x1080x9 two-dimensional Gaussian function values and equal number of fusion weights need to be calculated to generate an image, while using the method of the present application, only 60x60x9 two-dimensional Gaussian function values and equal number of fusion weights need to be calculated to generate an image, effectively reducing the calculation amount and complexity.

[0053] S4, image fusion, generating a multi-exposure fusion image:

[0054] S4.1, initializing i=1;

[0055] S4.2, initializing j=1;

[0056] S4.3, calculating the fusion pixel point gray value O ij (u,v) at the (u,v) position in the ith row and the jth column of the multi-exposure fusion image by using formula (3), and sequentially storing it in the SxT two-dimensional fusion gray value matrix B:

[0057]

[0058] In formula (3), I i+p,j+q(u,v) is the gray value of the pixel point at the (i+p)th row and (j+q)th column image block of the original image where the (i)th row and (j)th column image block is located;

[0059] S4.4, after j+1 is assigned to j, it is judged whether j≤16 is established, if yes, it returns to S4.3; otherwise, S4.5 is executed;

[0060] S4.5, after i+1 is assigned to i, it is judged whether i≤9 is established, if yes, it returns to S4.2; otherwise, S5 is executed;

[0061] S5, a 1920x1080 two-dimensional fused gray value matrix B is merged with a 1920x1080x2 three-dimensional color channel matrix A, thereby obtaining a 1920x1080x3 three-dimensional multi-exposure fused image and outputting.

[0062] In the embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0063] In the embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program is run by a processor to execute the steps of the above method.

Claims

1. An efficient multi-exposure image block-fusion method, characterized in that, The method comprises the following steps: S1, acquire Q original images with different exposure times, resolution of SxT and color standard of YCbCr, and divide each original image into image blocks of square with side length of N; Any image block in each original image is recorded as the image block in the i-th row and the j-th column, and S2, statistics of the original image in each image block gray distribution, and according to the original image in each image block gray distribution, calculate Q original image in the same position image block information entropy, so as to select the maximum information entropy corresponding to the image block, get A maximum information entropy image block and combined into a information entropy image; the information entropy image in each pixel point Cb, Cr two channel value is stored in SxTx2 three-dimensional color channel matrix A in turn; S3, calculating the fusion weight of each pixel in each image block in the information entropy image; S3.1, recording any one image block in the information entropy image as V, and recording the coordinates of any one pixel point r in the image block V as (u, v), u, v ∈ [0, N-1]; S3.2, centering on the image block V in the information entropy image and taking the image block as the image block in the 0th row and the 0th column, thus dividing an image block area with a dimension of HxH, and H is an odd number; taking any image block in the HxH image block area as the image block in the pth row and the qth column; and p, S3.3, calculating the fusion weight W of the pixel point r for the pixel point at the (u, v) position of the pth row qth column image block in the HxH image block region by using formula (1) p,q (u, v), so as to obtain the fusion weight of each pixel point of each image block in the information entropy image, and store the fusion weight into the three-dimensional fusion weight matrix U of N x N x H 2 . In formula (1), G m,n (u, v) represents the function value of the two-dimensional Gaussian function centered on the center pixel of the image block in the mth row and nth column in the H x H image block region at the (u, v) position in the image block V, G p,q (u, v) represents the function value of the two-dimensional Gaussian function centered on the center pixel of the image block in the pth row and qth column in the H x H image block region at the (u, v) position in the image block V, and has: In formula (2), d pq (u, v) is the distance between the pixel point at position (u, v) in the image block V and the center point of the image block in the pth row and qth column in the HxH image block region, and σ is the standard deviation of the two-dimensional Gaussian function. S4, performing image fusion to generate a multi-exposure fusion image: S4.1, initializing i = 1; S4.2, initializing j = 1; S4.3, calculate the fusion pixel point gray value O at the (i, v) position in the image block of the i-th row and the j-th column of the multi-exposure fusion image by using formula (3) ij (u, v), and sequentially stored in the two-dimensional fusion gray value matrix B of SxT: In formula (3), I i+p,j+q (u,v) is a gray value of a pixel point at a position (u,v) in an image block at an (i+p)th row and a (j+q)th column of an original image in which the image block at the ith row and the jth column is located. S4.4, after j+1 is assigned to j, it is judged whether it is established, if it is established, S4.3 is returned; Otherwise, perform S4.5; S4.5, after assigning i+1 to i, judging whether it is true, if true, returning to S4.2; Otherwise, perform S5; S5, merging the two-dimensional fusion gray value matrix B of SxT and the three-dimensional color channel matrix A of SxTx2, thereby obtaining a three-dimensional multi-exposure fusion image of SxTx3 and outputting.

2. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the multi-exposure image block fusion method of claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the multi-exposure image block fusion method of claim 1.

Images