Hdri image fusion method, system, storage medium and terminal

Abstract

The application provides an HDR image fusion method, system, storage medium and terminal, the HDR image fusion method comprises the following steps: acquiring device-related information and scene-related information of an LDR image, and generating a basic fusion function according to the device-related information and the scene-related information; calculating a first fusion weight of adjacent frame images in the LDR image through the basic fusion function; calculating a second fusion weight of all LDR images according to the first fusion weight; performing smoothing processing on the second fusion weight; determining a final fusion weight of each frame LDR image according to the second fusion weight after the smoothing processing, and fusing the LDR image according to the final fusion weight to obtain an HDR image. The application improves the image quality after image fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an HDR image fusion method, system, storage medium and terminal. BACKGROUND

[0002] In continuous bracketing exposure, the image dynamic range that can be captured by the sensor is usually limited, and in computational photography, a plurality of low dynamic range (LDR) images are usually collected by using different exposure information, and the LDR images of each exposure time are fused into a high dynamic range (HDR) image according to a certain proportion, so as to better reflect the visual effect in the real environment.

[0003] The longer the exposure time of the LDR image collected by the sensor, the higher the image brightness, and the dark area is visible but the bright area is overexposed; the shorter the exposure time, the lower the image brightness, and the dark area is invisible but the bright area is visible. In order to obtain an HDR image, the LDR raw bayer format data of different exposure times collected by the sensor is fused according to a proper proportion by using an HDR fusion algorithm, so as to obtain an HDR image with high dynamic range. In this process, the image brightness is evaluated according to the raw bayer format pixel value of the LDR image, the fusion weight of the raw bayer format image of different exposures is calculated by using a fusion weight calculation formula, and the raw bayer format pixel value is fused by weighting according to the corresponding weight, so as to generate a high dynamic range image.

[0004] In the current fusion process of the LDR image, a fusion weight calculation formula is usually designed to calculate the fusion weight by using image information, and then the LDR images of different exposure times are fused into an HDR image. However, there is a certain difference between the pixel relationship of the raw bayer format image and the real exposure parameter; the noise levels of the images of different exposure times are different; the edge of the image itself or the brightness difference in the fusion process is too large, and the quality of the fused image in these areas may be poor, so that the signal-to-noise ratio may be broken, the edge may be abnormal, and the fusion transition may be unnatural in the HDR fusion process, thereby affecting the imaging quality.

[0005] Therefore, it is necessary to provide a novel HDR image fusion method, system, storage medium and terminal to solve the above problems in the prior art. SUMMARY

[0006] The present application aims to provide an HDR image fusion method, system, storage medium and terminal, which improves the image quality after image fusion.

[0007] To achieve the above-mentioned purpose, the HDR image fusion method comprises:

[0008] obtaining device-related information and scene-related information of the LDR image, and generating a basic fusion function according to the device-related information and the scene-related information;

[0009] calculating first fusion weights of adjacent frame images in the LDR image through the basic fusion function;

[0010] calculating second fusion weights of the LDR image according to the first fusion weights;

[0011] smoothing the second fusion weights and calculating a guide coefficient of the LDR image;

[0012] determining final fusion weights of each frame of the LDR image according to the smoothed second fusion weights and the guide coefficient, and fusing the LDR image according to the final fusion weights to obtain an HDR image.

[0013] The HDR image fusion method has the advantages that the first fusion weights are calculated in the manner of adjacent frame image fusion, the model complexity and abnormal risk can be effectively reduced, the guide coefficient is calculated in the manner of guide filtering, the image edge details can be retained, part of the noise information in the image can be effectively removed, the final fused image can be smooth in transition and good in edge performance while the effective information is ensured, and the quality of the HDR fused image is improved.

[0014] Optionally, the first fusion weights of the adjacent frame images in the LDR image through the basic fusion function include:

[0015] taking a first frame of the LDR image and a second frame of the LDR image as a first adjacent frame pair, and obtaining first fusion weights of two images in the first adjacent frame pair according to the basic fusion function;

[0016] performing weighted fusion on the two images in the first adjacent frame pair according to the first fusion weights to obtain a second intermediate image; taking the second intermediate image and a third frame of the LDR image as a second adjacent frame pair, and obtaining first fusion weights of two images in the second adjacent frame pair according to the basic fusion function;

[0017] performing weighted fusion on the two images in the second adjacent frame pair according to the first fusion weights to obtain a third intermediate image;

[0018] The third frame intermediate image and the fourth frame LDR image are taken as the third adjacent frame pair, and the first fusion weights of the third frame intermediate image and the fourth frame LDR image are obtained respectively according to the basic fusion function;

[0019] The above process is repeated for subsequent LDR images until the first fusion weight of each frame of the LDR image is obtained;

[0020] In each of the adjacent frame pairs, the sum of the first fusion weights of the two frames is 1.

[0021] Optionally, the first fusion weight of the LDR image in the k-th frame k Satisfy the following formula:

[0022] weight k =f(LDR) k Noise(LDR) k-1 ),awbGain)

[0023] The intermediate image LDR′ in the kth frame k Satisfy the following formula:

[0024] LDR' k = (1-weight) k LDR k-1 +weight k LDR k

[0025] Among them, LDR′ k For the intermediate image of the k-th frame, weight k The first fusion weight of the k-th LDR image, LDR k Let f represent the k-th LDR image, and let Noise(LDR) represent the basic fusion function. k-1 ) represents the noise coefficient of the (k-1)th frame of the LDR image, awbGain represents the white balance coefficient of the LDR image, and k is an integer greater than 1.

[0026] Optionally, the smoothing process of the second fused weight information and the calculation of the steering coefficient of the LDR image include:

[0027] The second fusion weights of the N frames of the LDR images are downsampled and Gaussian filtered to obtain the low-resolution weights of each frame of the LDR images, where N is a positive integer;

[0028] The mean, correlation coefficient, and variance of pixels within the filtering window are calculated based on the low-resolution weights.

[0029] The first guide coefficient and the second guide coefficient of the N groups of low-resolution weight maps are calculated according to the pixel point mean value in the filter window, the correlation coefficient and the variance. The beneficial effect is that the first guide coefficient and the second guide coefficient of the N groups of low-resolution are calculated in a guided filter mode, which can improve the smoothness of the image and realize detail enhancement.

[0030] Optionally, the calculation process of the first guide coefficient and the second guide coefficient satisfies the following formula:

[0031]

[0032]

[0033] var k =corr k -mean k ·mean k

[0034] a k =var k / (var k +ε)

[0035] b k =mean k -a k ·mean k

[0036] wherein, mean k is the pixel point mean value in the filter window, corr k is the correlation coefficient, var k is the variance, a k is the first guide coefficient, b k is the second guide coefficient, f mean is a mean filter function for calculating mean, ε is a constant, and k is an integer from 1 to N.

[0037] Optionally, the final fusion weight of each frame of the LDR image is determined according to the second fusion weight after the smoothing processing, and the LDR image is fused according to the final fusion weight to obtain an HDR image, comprising:

[0038] The first guide coefficient and the second guide coefficient of the N groups of low-resolution weight maps are up-sampled to the original resolution of the LDR image to obtain N groups of third guide coefficients and fourth guide coefficients of the original resolution, respectively;

[0039] The third guide coefficient and the fourth guide coefficient are used to map the second fusion weight to obtain a third fusion weight;

[0040] normalizing the third fusion weight of the N frames of LDR images to obtain a target fusion weight;

[0041] performing weighted fusion on the N frames of LDR images according to the target fusion weight to obtain an HDR image.

[0042] Optionally, the calculation process of the third fusion weight satisfies the following formula:

[0043]

[0044] wherein k is an integer from 1 to N, is the third fusion weight of the kth frame of LDR images, weight′ k is the second fusion weight of the kth frame of LDR images, a′ k is the third guide coefficient, b′ k is the fourth guide coefficient.

[0045] The calculation process of the target fusion weight satisfies the following formula:

[0046]

[0047]

[0048] wherein k is an integer from 1 to N, and sum is the sum of the third fusion weights of the N frames of LDR images.

[0049] The application further provides an HDR image fusion device, comprising:

[0050] a function generation module configured to acquire device-related information and scene-related information of LDR images, and generate a basic fusion function according to the device-related information and the scene-related information;

[0051] a first weight calculation module configured to calculate first fusion weights of adjacent frame images in the LDR images by using the basic fusion function;

[0052] a second weight calculation module configured to calculate second fusion weights between each frame image in the LDR images according to the first fusion weights;

[0053] a coefficient calculation module configured to perform smoothing processing on the second fusion weights, and calculate guide coefficients of the LDR images;

[0054] a fusion processing module configured to determine final fusion weights of each frame of the LDR images according to the second fusion weights after the smoothing processing and the guide coefficients, and perform fusion on the LDR images according to the final fusion weights to obtain an HDR image.

[0055] The application further discloses a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the HDR image fusion method.

[0056] The application further provides a terminal, comprising a processor and a memory.

[0057] The memory is used for storing a computer program.

[0058] The processor is used for executing the computer program stored in the memory, so that the terminal executes the HDR image fusion method. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 A flow chart of the HDR image fusion method of the embodiment of the application.

[0060] Figure 2 A curve diagram of function f1 in the HDR image fusion method of the embodiment of the application.

[0061] Figure 3 A flow chart of step S103 in the HDR image fusion method of the embodiment of the application.

[0062] Figure 4 A flow chart of step S104 in the HDR image fusion method of the embodiment of the application.

[0063] Figure 5 A structure block diagram of the HDR image fusion device of the embodiment of the application. DETAILED DESCRIPTION

[0064] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the common meanings of the technical terms or scientific terms by those skilled in the art. The similar words such as "comprise" used herein mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, and do not exclude other elements or objects.

[0065] In view of the problems in the prior art, the embodiments of the application provide an HDR image fusion method, which refers to Figure 1 and comprises the following steps:

[0066] S101, obtaining device-related information and scene-related information of an LDR image, and generating a basic fusion function according to the device-related information and the scene-related information.

[0067] In order to improve the fusion effect of the LDR image, the basic fusion function of the HDR image is calibrated according to the device and the application scene-related information. First, the model of the basic function f is determined, including its input and output, function type, adjustment parameter and fixed parameter, etc. The model of the function can be a broken line, a curve or a multi-dimensional fold surface, and the broken line and the fold surface control the inflection points through the extracted device-related information and the scene-related information to obtain the best function. The curve and the surface set a smooth curve or surface, and adjust the extreme point and the slope to obtain the best function.

[0068] For example, in a multi-dimensional calculation manner, the basic fusion function is constructed by considering the influence of the pixel value, the noise model and the white balance parameter in the scene information, so that the information of the image and the device characteristics are better utilized, the calculation of the subsequent fusion weight is more reasonable, the noise and the color influence are considered at the same time of ensuring the dynamic range.

[0069] Specifically, the noise model and the white balance coefficient of the photographing device of the LDR image are calibrated, and the basic fusion function is designed as:

[0070] weight=f(l,Noise(s),awbGain)=f2(f1(Y),noise)

[0071]

[0072] noise=Noise(s)

[0073] Wherein, weight∈[0,1] represents the first fusion weight of the LDR image, the LDR image includes a long exposure frame image and a short exposure frame image, l is the current point pixel value of the long exposure frame image, s is the current point pixel value of the short exposure frame image, and awbGain is the white balance coefficient. is the image information of the current point, and is a function about the pixel value x and the awbGain white balance coefficient; f1(Y) is a fusion weight curve which is monotonically decreasing about the image information Y; noise=Noise(s) is the noise model of the LDR image photographing device, which is a function monotonically non-decreasing about the short exposure frame pixel value s, and f2(f1(Y),noise) is a function monotonically increasing about the noise of the short exposure frame. Wherein, the function According to the application scene, the device characteristics and the application requirements, there are different calculation methods, for example, extracting the gradient in different directions By combining the gradients in each direction with the current pixel value, a function for strengthening the boundary can be obtained. And thus obtain the corresponding Y value.

[0074] For example, in some devices, the noise model is

[0075]

[0076] Where a and b are constants, and s is the current pixel value of the short exposure frame image.

[0077] The function f1 can be expressed as a simple piecewise line or a smooth curve, such as the sigmoid function, which is approximately symmetric about y = 0.5 over a finite interval. Figure 2 As shown, the formula is as follows:

[0078]

[0079] The function f2 is designed based on different noise models and scenarios, for example...

[0080] f2(x,y) = 1 - y + xy

[0081] Where x,y∈[0,1], x and y represent the two inputs of function f2 respectively.

[0082] S102. Calculate the first fusion weight of adjacent frames in the LDR image using the basic fusion function.

[0083] In some embodiments, reference Figure 3 The process of step S102 includes:

[0084] S301. Take the first frame of the LDR image and the second frame of the LDR image as the first adjacent frame pair, and obtain the first fusion weights of the two frames in the first adjacent frame pair according to the basic fusion function.

[0085] S302. The two frames of the first adjacent frame group are weighted and fused according to the first fusion weight of the two frames in the first adjacent frame pair to obtain the second intermediate frame image; the second intermediate frame image and the third LDR image are taken as the second adjacent frame pair, and the first fusion weight of the two frames in the second adjacent frame pair is obtained according to the basic fusion function.

[0086] S303. The two images in the second adjacent frame pair are weighted and fused according to the first fusion weight of the two images in the second adjacent frame pair to obtain the third intermediate image;

[0087] S304. Take the third frame intermediate image and the fourth frame LDR image as the third adjacent frame pair, and obtain the first fusion weights of the third frame intermediate image and the fourth frame LDR image according to the basic fusion function;

[0088] S305. The above process is repeated for subsequent LDR images until the first fusion weight of each frame of the LDR image is obtained;

[0089] In each of the adjacent frame pairs, the sum of the first fusion weights of the two frames is 1.

[0090] For example, firstly, the first fusion weight weight2 of LDR2 and the first fusion weight (1-weight2) of LDR1 are calculated using the second LDR image LDR2 and the first LDR image LDR1, that is...

[0091] weight2=f(LDR2,Noise(LDR1),awbGain)

[0092] The first and second LDR frames are weighted and fused according to the first fusion weight to obtain the intermediate image LDR′2 of the fused second LDR frame. Then, the first fusion weight weight3 of LDR3 is calculated based on the intermediate image and the third LDR frame LDR3. Subsequent LDR images are fused pairwise in this way to obtain weight3 and LDR′3, ..., after accumulating N-3 times, N-1 weighted images weight2, weight3, ..., weight3 are obtained from the third LDR frame to the Nth LDR frame. N Each weight map k ∈[0,1], k is an integer from 2 to N. By using a two-frame-by-frame fusion method, the model complexity and anomaly risk can be reduced.

[0093] In some embodiments, the first fusion weight of the LDR image in the k-th frame k Satisfy the following formula:

[0094] weight k =f(LDR) k Noise(LDR′) k-1 ),awbGain)

[0095] The intermediate image LDR′ in the kth frame k Satisfy the following formula:

[0096] LDR' k = (1-weight) k LDR k-1+ weight k • LDR k

[0097] wherein LDR'k k is the intermediate image in the kth frame, weight k is the first fusion weight of the kth LDR image, LDR k denotes the kth LDR image, f denotes the basic fusion function, Noise(LDR k-1 ) denotes the noise coefficient of the k-1th LDR image, awbGain denotes the white balance coefficient of the LDR image, and k is an integer greater than 1.

[0098] S103, calculating a second fusion weight of the LDR image according to the first fusion weight.

[0099] Using N-1 groups of first fusion weights, the second fusion weights weight' k corresponding to all LDR images are reconstructed. Since each weight k only embodies the fusion relationship between two frames, it cannot represent the respective proportion weight' k of N original images in the final fusion image, but weight' k can be inversely deduced from weight k , so weight' k can be reconstructed accordingly, k = 1, 2, …, N.

[0100] The reconstruction method is as follows:

[0101] HDR = weight'1·LDR1+ weight'2·LDR2+ … + weight' N ·LDR N

[0102] wherein:

[0103] weight'1= (1-weight2) (1-weight3) … (1-weight N )

[0104] weight'2= weight2 (1-weight3) … (1-weight N )

[0105] weight'2= weight3 (1-weight4) … (1-weight N )

[0106] …

[0107] weight'N-1 = weight N-1 (1-weight N )

[0108] weight′ N = weight N

[0109] Similarly, the second fusion weight weight′ k of the kth frame LDR image is known as follows: k is an integer from 1 to N.

[0110] S104, smoothing the second fusion weight and calculating the guide coefficient of the LDR image.

[0111] After obtaining the second fusion weight of the N frames of LDR images, the N-N fusion weights are smoothed, and the details and edge regions are enhanced, so that the image fusion is smoother while retaining and enhancing the details.

[0112] In this embodiment, the method of extracting multi-level information, smoothing and detail enhancement is Laplacian pyramid; using Fourier transform image conversion to frequency domain, separating high and low frequencies of the image; resolution sampling, combined with the calculation method of guide filter to realize the smoothing and detail enhancement of multiple images.

[0113] For example, the resolution sampling method of guide filter can effectively protect the image detail part, and the process is as follows:

[0114] Down-sampling the second fusion weight of the N frames of LDR images and performing Gaussian filtering to obtain a low-resolution weight of each frame of the LDR image, N is a positive integer;

[0115] According to the low-resolution weight, the mean, correlation coefficient and variance of the pixel points in the filter window are calculated respectively;

[0116] According to the mean, correlation coefficient and variance of the pixel points in the filter window, the first guide coefficient and the second guide coefficient of N groups of low-resolution weight maps are calculated.

[0117] Specifically, by taking N second fusion weights weight′1, weight′2, …, weight′ N Down-sampling and Gaussian filtering to obtain low-resolution weights Then taking them as input images and guide images at the same time to obtain the first guide coefficient a k and the second guide coefficient b kIn the embodiment, the filter window is a 3*3 window, and the present solution is not particularly limited to this.

[0118] In some embodiments, the first guide coefficient a k and the second guide coefficient b k are calculated according to the following formula:

[0119]

[0120]

[0121] var k =corr k -mean k ·mean k

[0122] a k =var k / (var k +ε)

[0123] b k =mean k -a k ·mean k

[0124] wherein mean k is the average of the pixel points in the filter window, corr k is the correlation coefficient, var k is the variance, a k is the first guide coefficient, b k is the second guide coefficient, f mean is the average filtering function for calculating mean, ε is a constant, and k is an integer from 1 to N.

[0125] S105, determining the final fusion weight of each frame of the LDR image according to the second fusion weight after smoothing processing and the guide coefficient, and fusing the LDR image according to the final fusion weight to obtain an HDR image.

[0126] In some embodiments, with reference Figure 4 to the process of step S105, the process includes:

[0127] S401, upsampling the first guide coefficient and the second guide coefficient of the N groups of low-resolution weight maps to the original resolution of the LDR image to obtain N groups of third guide coefficients and fourth guide coefficients of the original resolution, respectively;

[0128] S402, mapping the second fusion weight according to the third guide coefficient and the fourth guide coefficient to obtain a third fusion weight;

[0129] S403, normalizing the third fusion weight of N frames of LDR images to obtain a target fusion weight;

[0130] S404, performing weighted fusion on the N frames of LDR images according to the target fusion weight to obtain an HDR image.

[0131] In some embodiments, the calculation process of the third fusion weight satisfies the following formula:

[0132]

[0133] wherein k is an integer from 1 to N, weight′k is the third fusion weight of the kth frame of LDR images, k a′k is the second fusion weight of the kth frame of LDR images, k a′k is the third guide coefficient, k b′k is the fourth guide coefficient.

[0134] Specifically, in combination with the resolution sampling method of guide filtering, first, N groups of guide coefficients a k and b k are up-sampled to the original resolution to obtain third guide coefficients a′ k and fourth guide coefficients b′ k (k = 1, 2 … N), and weight′1, weight′2, …, weight′ N are mapped with respect to a′ k and b′ k , that is:

[0135]

[0136] wherein k is an integer from 1 to N.

[0137] Then, the third fusion weight weight′k of each frame of LDR images is obtained by normalization, that is,

[0138]

[0139]

[0140] wherein k is an integer from 1 to N, and sum is the sum of the third fusion weights of N frames of LDR images.

[0141] Finally, the above weight is used to perform weighted fusion on the original image to obtain an HDR image, that is, ​

[0142]

[0143] wherein LDR N denotes the Nth frame of LDR image, denotes the third fusion weight of the Nth frame of LDR image.

[0144] The scheme realizes the HDR fusion algorithm, generates a high dynamic range image, guarantees the effective information of the image, makes the image transition smooth and the edge performance good, and improves the quality of the HDR fusion image. The scheme firstly uses a multi-dimensional design basic fusion function, including pixel value, noise model, white balance parameter, etc., can better utilize the information of the image and the device characteristics, makes the calculation of the fusion weight more reasonable, guarantees the dynamic range, also considers the influence of noise and color, and can be suitable for more extensive and complex scenes. Secondly, the fusion weight is calculated frame by frame and fused, and then the operation is repeated with the next frame, the weight calculated each time is stored, and a series of processing is performed on the weight, the weight of all frames is obtained through the fusion weight of two frames, and the weight is normalized, then the new weight is down-sampled to obtain a low-resolution small image, the guide coefficient of the small image is calculated, and the small image is up-sampled to filter the original resolution image, and a new fusion weight that is smooth and protects effective details is obtained, finally, the target image is obtained through weighted fusion. Different from the traditional guide filtering, the idea of guide filtering is used to calculate different information on images of different resolutions, so that the problems of image transition not smooth, abnormal fusion edge and signal-to-noise ratio rupture can be improved.

[0145] The application further discloses an HDR image fusion device, referring to Figure 5 , comprising:

[0146] The function generation module 501 is used for acquiring device-related information and scene-related information of the LDR image, and generating a basic fusion function according to the device-related information and the scene-related information.

[0147] The first weight calculation module 502 is used for calculating the first fusion weight of adjacent frame images in the LDR image through the basic fusion function.

[0148] The second weight calculation module 503 is used for calculating the second fusion weight of the LDR image according to the first fusion weight.

[0149] The coefficient calculation module 504 is used for performing smoothing processing on the second fusion weight and calculating the guide coefficient of the LDR image.

[0150] The fusion processing module 505 is configured to determine a final fusion weight of each frame of the LDR image according to the smoothed second fusion weight and the guide coefficient, and fuse the LDR image according to the final fusion weight to obtain an HDR image.

[0151] It should be noted that the structure and principle of the above-described HDR image fusion device correspond to the steps in the above-described HDR image fusion method one by one, and thus will not be described here again.

[0152] It should be understood that the division of each module of the above device is only a logical division of functions, and all or part of the modules can be integrated into one physical entity, or can be physically separated. These modules can all be implemented in the form of software called by a processing element; all can be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the selection module can be a separately established processing element, or can be integrated into a chip of the above-described system, in addition, it can also be stored in the form of program code in the memory of the above-described system, and the function of the above x module can be called and executed by a processing element of the above-described system. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or can be independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of hardware or the instruction of software in the processing element.

[0153] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of program code called by a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together to implement in the form of a system on a chip (SOC).

[0154] The application further discloses a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0155] The storage medium of the application stores a computer program, and the computer program is executed by a processor to realize the method described above. The storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk, a U disk, a memory card, an optical disk and various storage medium capable of storing program codes.

[0156] The application further discloses a terminal, which comprises a processor and a memory.

[0157] The memory is used for storing a computer program.

[0158] The processor is used for executing the computer program stored in the memory, so that the terminal executes the HDR image fusion method described above.

[0159] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0160] The functional units in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0161] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or in other words the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0162] Although the embodiments of the present application are described in detail above, it is obvious to those skilled in the art that various modifications and changes can be made to the embodiments. However, it should be understood that such modifications and changes are within the scope and spirit of the present application described in the claims. Moreover, the present application described herein can have other embodiments and can be implemented or realized in various ways.

Claims

1. A method of HDR image fusion, characterized in that, The method comprises the following steps: obtaining device-related information and scene-related information of an LDR image, and generating a basic fusion function according to the device-related information and the scene-related information; calculating first fusion weights of adjacent frame images in the LDR image through the basic fusion function; calculating second fusion weights of the LDR image according to the first fusion weights; performing smoothing processing on the second fusion weights, comprising: performing down-sampling processing on the second fusion weights of N frames of the LDR image and performing Gaussian filtering processing to obtain low-resolution weights of each frame of the LDR image, N being a positive integer; calculating pixel point mean values, correlation coefficients and variances in a filter window according to the low-resolution weights respectively; calculating first guide coefficients and second guide coefficients of N groups of low-resolution weight maps according to the pixel point mean values, the correlation coefficients and the variances in the filter window; determining final fusion weights of each frame of the LDR image according to the second fusion weights after the smoothing processing, and fusing the LDR image according to the final fusion weights to obtain an HDR image, comprising: up-sampling the first guide coefficients and the second guide coefficients of the N groups of low-resolution weight maps to the original resolution of the LDR image to obtain third guide coefficients and fourth guide coefficients of N groups of original resolution respectively; performing mapping on the second fusion weights according to the third guide coefficients and the fourth guide coefficients to obtain third fusion weights; performing normalization processing on the third fusion weights of N frames of LDR images to obtain target fusion weights; and performing weighted fusion on N frames of the LDR image according to the target fusion weights to obtain an HDR image.

2. The HDR image fusion method of claim 1, wherein, The calculation of the first fusion weights of the adjacent frame images in the LDR image through the basic fusion function comprises: taking a first frame of the LDR image and a second frame of the LDR image as a first adjacent frame pair, and obtaining first fusion weights of two images in the first adjacent frame pair according to the basic fusion function respectively; performing weighted fusion on the two images in the first adjacent frame pair according to the first fusion weights of the two images to obtain a second intermediate image; taking the second intermediate image and a third frame of the LDR image as a second adjacent frame pair, and obtaining first fusion weights of two images in the second adjacent frame pair according to the basic fusion function respectively; performing weighted fusion on the two images in the second adjacent frame pair according to the first fusion weights of the two images to obtain a third intermediate image; taking the third intermediate image and a fourth frame of the LDR image as a third adjacent frame pair, and obtaining first fusion weights of the third intermediate image and the fourth frame of the LDR image according to the basic fusion function respectively; performing the above process on subsequent LDR images in a loop until the first fusion weights of each frame of the LDR image are obtained; wherein the sum of the first fusion weights of the two images in each adjacent frame pair is 1.

3. The HDR image fusion method of claim 2, wherein, the first blending weight of the LDR image of the kth frame satisfies the following equation: the intermediate image of the kth frame satisfies the following equation: wherein, is an intermediate image in the kth frame, is a first fusion weight for the kth frame LDR image, denotes the kth frame LDR image, f denotes the base fusion function, denotes a noise coefficient for the k-1th frame LDR image, denotes a white balance coefficient for the LDR image, k is an integer greater than 1.

4. The HDR image fusion method of claim 1, wherein, The calculation process of the first guide coefficients and the second guide coefficients satisfies the following formula: wherein, is a mean value of the pixels within the filter window, is a correlation coefficient, is a variance, is the first steering coefficient, is the second steering coefficient, is a mean filter function for calculating mean, is a constant, k is an integer from 1 to N.

5. The HDR image fusion method of claim 1, wherein, The calculation process of the third fusion weight satisfies the following formula: wherein, is an integer, is a third fusion weight for the kth frame LDR image, is a second fusion weight for the kth frame LDR image, is the third steering coefficient, is the fourth steering coefficient; The calculation process of the target fusion weight satisfies the following formula: wherein, is the sum of the third fusion weights for the N frames of LDR images.

6. An HDR image fusion apparatus for implementing the HDR image fusion method according to any one of claims 1 to 5, characterized in that, The HDR image fusion device comprises: A function generation module is configured to acquire device-related information and scene-related information of an LDR image, and generate a basic fusion function according to the device-related information and the scene-related information; A first weight calculation module is configured to calculate a first fusion weight of adjacent frame images in the LDR image by using the basic fusion function; A second weight calculation module is configured to calculate a second fusion weight of the LDR image according to the first fusion weight; A coefficient calculation module is configured to perform smoothing processing on the second fusion weight, and calculate a guide coefficient of the LDR image; A fusion processing module is configured to determine a final fusion weight of each frame of the LDR image according to the second fusion weight after the smoothing processing and the guide coefficient, and perform fusion on the LDR image according to the final fusion weight to obtain an HDR image.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by a processor to implement the HDR image fusion method in any one of claims 1 to 5.

8. A terminal, characterized by comprising: Comprise: A processor and a memory; The memory is configured to store a computer program; The processor is configured to execute the computer program stored in the memory, so that the terminal executes the HDR image fusion method in any one of claims 1 to 5.

Images