Image fusion method, electronic device, and storage medium

Abstract

Embodiments of the present application disclose an image fusion method, an electronic device and a storage medium. The method comprises: acquiring a target image sequence, the target image sequence comprising a plurality of to-be-fused images of different exposure degrees in the same scene; determining an initial weight image of each to-be-fused image according to an index parameter of a pixel point of each to-be-fused image; determining a to-be-corrected region in a to-be-corrected initial weight image, the to-be-corrected initial weight image being an initial weight image corresponding to a to-be-fused image of the highest exposure degree, and the to-be-corrected region being a region of a corresponding position of a white object in the to-be-fused image of the highest exposure degree on the to-be-corrected initial weight image; improving a weight value of each pixel point in the to-be-corrected region to obtain a target weight image; and fusing the to-be-fused image of the highest exposure degree and other to-be-fused images according to the target weight image and initial weight images of the other to-be-fused images except the to-be-fused image of the highest exposure degree, to obtain a target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image fusion method, an electronic device and a storage medium. BACKGROUND

[0002] Multi-exposure image fusion (MEF) is a process of fusing a high-definition and color detail-rich image by using three or more images of the same scene with different exposure levels and performing some image processing operations in the image transform domain or spatial domain.

[0003] In the related art, first, a weight image of each to-be-fused image is constructed based on the contrast, saturation and brightness of each pixel point in the to-be-fused image, and then all to-be-fused images are fused based on the weight image of each to-be-fused image to obtain a final fusion result.

[0004] However, in the related art, the idea of good exposure is adopted, that is, a pixel point close to a preset good exposure value is given a larger brightness value, and a pixel point far from the preset good exposure value is given a smaller brightness value. Therefore, for a white object such as a white wall or a white clothes whose real scene brightness value is high, the white object will be darkened together with an overexposed area during fusion, resulting in poor image quality of multi-exposure fusion. SUMMARY

[0005] Embodiments of the present application provide an image fusion method, an electronic device and a storage medium to solve the technical problem of darkening of an image part during multi-exposure image fusion in the related art.

[0006] According to a first aspect of the present application, an image fusion method is disclosed, the method comprising:

[0007] obtaining a target image sequence, wherein the target image sequence comprises a plurality of to-be-fused images of different exposure levels of the same scene;

[0008] determining an initial weight image of each to-be-fused image according to an index parameter of a pixel point of each to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation and brightness;

[0009] determining a to-be-corrected region in a to-be-corrected initial weight image, wherein the to-be-corrected initial weight image is an initial weight image corresponding to a to-be-fused image with the highest exposure level in the target image sequence, and the to-be-corrected region is a region of a corresponding position of a white object in the to-be-fused image with the highest exposure level on the to-be-corrected initial weight image;

[0010] increase the weight value of each pixel in the to-be-corrected region, and keep the weight value of each pixel in other regions outside the to-be-corrected region in the initial weight image of the to-be-corrected unchanged, to obtain a target weight image;

[0011] According to the target weight image and the initial weight image of other to-be-fused images in the target image sequence except the to-be-fused image with the highest exposure, the to-be-fused image with the highest exposure and the other to-be-fused images are fused to obtain a target image.

[0012] According to a second aspect of the present application, an image fusion device is disclosed, the device comprising:

[0013] An acquisition module is configured to acquire a target image sequence, wherein the target image sequence comprises a plurality of to-be-fused images with different exposures under the same scene.

[0014] A first determination module is configured to determine an initial weight image of each to-be-fused image according to an index parameter of each pixel in the to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation and brightness.

[0015] A second determination module is configured to determine a to-be-corrected region in the initial weight image to be corrected, wherein the initial weight image to be corrected is an initial weight image corresponding to a to-be-fused image with the highest exposure in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure on the initial weight image to be corrected.

[0016] A correction module is configured to increase the weight value of each pixel in the to-be-corrected region, and keep the weight value of each pixel in other regions outside the to-be-corrected region in the initial weight image of the to-be-corrected unchanged, to obtain a target weight image.

[0017] A fusion module is configured to fuse the to-be-fused image with the highest exposure and other to-be-fused images in the target image sequence according to the target weight image and the initial weight image of the other to-be-fused images except the to-be-fused image with the highest exposure, to obtain a target image.

[0018] According to a third aspect of the present application, an electronic device is disclosed, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the image fusion method in the first aspect.

[0019] According to a fourth aspect of the present application, a computer readable storage medium is disclosed, which stores a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the image fusion method in the first aspect.

[0020] According to a fifth aspect of the present application, a computer program product is disclosed, comprising computer programs / instructions which, when executed by a processor, implement the image fusion method as in the first aspect.

[0021] In the embodiments of the present application, a target image sequence is acquired, wherein the target image sequence comprises a plurality of to-be-fused images of different exposure degrees under the same scene; an initial weight image of each to-be-fused image is determined according to an index parameter of a pixel point of each to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation and brightness; a to-be-corrected region in the initial weight image is determined, wherein the initial weight image to be corrected is an initial weight image corresponding to a to-be-fused image of the highest exposure degree in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image of the highest exposure degree on the initial weight image to be corrected; a weight value of each pixel point in the to-be-corrected region is improved, and a weight value of each pixel point in other regions outside the to-be-corrected region in the initial weight image to be corrected is kept unchanged to obtain a target weight image; and the to-be-fused image of the highest exposure degree and other to-be-fused images are fused according to the target weight image and initial weight images of the other to-be-fused images except the to-be-fused image of the highest exposure degree in the target image sequence to obtain a target image.

[0022] It can be seen that in the embodiments of the present application, the weight value of the white object in the to-be-fused image of the highest exposure degree during fusion can be improved, and since the brightness of the white object in the to-be-fused image of the highest exposure degree is the highest, improving the weight value of the white object in the to-be-fused image of the highest exposure degree during fusion can correct the darkening of the non-overexposure region where the white object is caused by multi-exposure fusion, and improve the image quality of multi-exposure fusion. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is an example diagram of a plurality of to-be-fused images of different exposure degrees under the same scene provided by the embodiments of the present application;

[0024] Figure 2 is an example diagram of a multi-image fusion process based on a pyramid strategy provided by the embodiments of the present application;

[0025] Figure 3 is a flowchart of an image fusion method provided by the embodiments of the present application;

[0026] Figure 4 is an example diagram of an image fusion method provided by the embodiments of the present application;

[0027] Figure 5 is a flowchart of one embodiment of step 303 provided by the embodiments of the present application;

[0028] Figure 6 is an example diagram of a cumulative histogram of different exposure images provided by an embodiment of the present application;

[0029] Figure 7 is a structural schematic diagram of an image fusion device provided by an embodiment of the present application;

[0030] Figure 8 is a structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the above objectives, characteristics and advantages of the present application more apparent, comprehensible and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0032] It should be noted that, for the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily required by the present application.

[0033] In recent years, important progress has been made in the research of computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence. Artificial intelligence (AI) is a new science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline that involves chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks and many other technology categories. Computer vision, as an important branch of artificial intelligence, is specifically to enable machines to recognize the world. Computer vision technology generally includes face recognition, liveness detection, fingerprint recognition and anti-forgery verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, behavior recognition, three-dimensional reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning, and other technologies. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security, city management, traffic management, building management, park management, face passage, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile imaging, cloud services, smart home, wearable devices, driverless vehicles, autonomous driving, intelligent medical care, face payment, face unlocking, fingerprint unlocking, face and certificate verification, smart screens, smart televisions, cameras, mobile Internet, network live streaming, beauty, makeup, medical cosmetology, intelligent temperature measurement, and other fields.

[0034] Taking multi-exposure fusion in the field of image processing as an example, multi-exposure fusion utilizes multiple images of different exposure levels under the same scene to fuse into an image with high clarity and rich colors through a series of image processing. In related technologies, first, a weight image of each to-be-fused image is constructed based on the contrast, saturation and brightness information of each pixel point in the to-be-fused image. Then, all to-be-fused images are fused based on the weight images of the to-be-fused images to obtain a final fusion result.

[0035] However, because the related technology adopts the idea of good exposure, a pixel point close to a preset good exposure value is given a larger brightness value, and a pixel point far from the preset good exposure value is given a smaller brightness value. Therefore, for a white object such as a white wall or a white clothes, which has a high brightness value in a real scene, the white object will be darkened together with an overexposed area during fusion.

[0036] For example, the preset good exposure value is 120, as shown in Figure 1 Figure 1 ​The image includes three images of the same scene with different exposures: P1, P2, and P3. In P1, the exposure value of the white wall M1 is 187; in P2, it's 122; and in P3, it's 80. Since the exposure value of white wall M2 in P2 is close to a preset good exposure value, it is assigned a larger brightness value. Conversely, since the exposure value of white wall M1 in P1 is far from the preset good exposure value, it is assigned a smaller brightness value. Similarly, since the exposure value of white wall M3 in P3 is also far from the preset good exposure value, it is assigned a smaller brightness value. When fusing images based on these assigned brightness values, the white walls and overexposed areas are darkened together, resulting in poor image quality from multi-exposure fusion.

[0037] To address the aforementioned technical problems, embodiments of this application provide an image fusion method, an electronic device, and a storage medium. For ease of understanding, the application scenarios and some related concepts of these embodiments are first introduced below.

[0038] Multi-Exposure Fusion (MEF): This process combines three or more images of the same scene at different exposures, performing image processing operations in the image transform domain or spatial domain, to create a single image with high clarity and rich color detail. For example, Figure 1 The image consists of three images with different exposures in the same scene, namely image P1, image P2, and image P3 to be fused. Using images P1, P2, and P3, a target image with high clarity and rich color details is fused together.

[0039] Pyramid fusion strategy: The input image is decomposed using a Laplacian pyramid, and the weight image of the input image is decomposed using a Gaussian pyramid. Based on the multi-scale weight image obtained from the Gaussian pyramid decomposition, the corresponding scale input images obtained from the Laplacian pyramid decomposition are fused. Finally, the final fusion result is obtained by reconstructing the Laplacian pyramid.

[0040] For example, with Figure 1 Taking the images P1, P2, and P3 to be fused as examples, calculate the weight image W1 of image P1, the weight image W2 of image P2, and the weight image W3 of image P3.

[0041] Then as Figure 2As shown, the Laplacian pyramid decomposition is used to decompose the to-be-fused images P1, P2, P3 to obtain a plurality of decomposition images of different scales of each to-be-fused image, for example, the Laplacian pyramid decomposition images of the to-be-fused image P1 are L1{P1}, L2{P1}, …, L M {P1}, the Gaussian pyramid decomposition is used to decompose the weight images W1, W2, W3 to obtain a plurality of decomposition images of different scales of each weight image, for example, the Gaussian pyramid decomposition images of the weight image W1 are G1{W1}, G2{W1}, …, G M {W1}, wherein M is the number of scales of the pyramid decomposition image, and M is an integer greater than 1.

[0042] Then, based on the decomposition images of the same scale of the Gaussian pyramid, the decomposition images of the same scale of the Laplacian pyramid are fused to obtain a fused pyramid image sequence R1, R2, …, R M ; wherein,

[0043] R1=L1{P1}*G1{W1}+L1{P2}*G1{W2}+L1{P3}*G1{W3},

[0044] R2=L2{P1}*G2{W1}+L2{P2}*G2{W2}+L2{P3}*G2{W3},

[0045] R M =L M 1{P1}*G M 1{W1}+L M 2{P2}*G M 2{W2}+L M 3{P3}*G M 3{W3}, finally, the R1,

[0046] R2, …, R M are reconstructed by the Laplacian pyramid to obtain a final fusion result.

[0047] Cumulative histogram: represents the cumulative probability distribution of the image composition components at the gray level, and each probability value represents the probability of being less than or equal to the gray value.

[0048] Next, an image fusion method provided by an embodiment of the present application is introduced.

[0049] Figure 3 is a flowchart of an image fusion method provided by an embodiment of the present application, as shown in the figure, the method can include the following steps: step 301, step 302, step 303, step 304 and step 305. Figure 3

[0050] ​In step 301, a target image sequence is acquired, wherein the target image sequence includes a plurality of to-be-fused images with different exposure degrees in the same scene.

[0051] In the embodiments of the present application, the plurality of to-be-fused images are fused to obtain a target image which contains both bright and dark parts of the scene and has no loss of details.

[0052] In one example, the target image sequence can include three to-be-fused images with different exposure degrees, i.e., an underexposed to-be-fused image, a well-exposed to-be-fused image, and an overexposed to-be-fused image.

[0053] In step 302, an initial weight image of each to-be-fused image is determined according to an index parameter of each pixel point of the to-be-fused image, wherein the index parameter includes at least one of contrast, saturation, and brightness.

[0054] In the embodiments of the present application, the initial weight image contains a weight value of each pixel point of the to-be-fused image in the fusion, wherein the weight value in the fusion is also referred to as a fusion weight value for convenience of description.

[0055] In some embodiments of the present application, the contrast C of the pixel point in the image is calculated in the following manner: generally, the edges of objects in underexposed or overexposed areas are difficult to be detected, and the Laplacian operator L = [0, -1, 0; -1, 4, -4; 0, -1, 0] can be used to calculate the contrast C on the Y channel of the image, and a larger weight value is given to the edge pixel: C = |L * Y|.

[0056] In some embodiments of the present application, the saturation S of the pixel point in the image is calculated in the following manner: a well-exposed pixel point can capture the color saturation well, and in the RGB color space, the standard deviation of each pixel point in the R, G, and B channels can be used as the metric S; in the YUV color space, the calculation formula of S is as follows: S = |U| + |V| + 1.

[0057] In some embodiments of the present application, the brightness E of the pixel point in the image is calculated in the following manner: a general idea is that the brightness of a well-exposed pixel point tends to be close to 0.5 with a high probability, and the calculation formula of E is as follows: wherein Y is the gray value of the pixel point in the image, and μ and δ are artificial set values.

[0058] In some embodiments of the present application, in order to improve the signal-to-noise ratio while retaining the detail information of the dark area, the index parameter can further include a quality metric, wherein the quality metric is positively correlated with the gray value of the pixel point, the quality metric is used to balance the brightness of the image in the image fusion process, and the quality metric

[0059] In the embodiment of the present application, when calculating the fusion weight value of each pixel point in the image to be fused, each item in the index parameter of the pixel point is multiplied to obtain the fusion weight value of the pixel point.

[0060] For example, if the index parameter includes contrast, saturation, and brightness, the fusion weight value of each pixel point in the image to be fused = contrast of the pixel point * saturation of the pixel point * brightness of the pixel point.

[0061] For example, if the index parameter includes contrast, saturation, brightness, and quality metric, the fusion weight value of each pixel point in the image to be fused = contrast of the pixel point * saturation of the pixel point * brightness of the pixel point * quality metric of the pixel point.

[0062] In step 303, a to-be-corrected region in the initial weight image to be corrected is determined, wherein the initial weight image to be corrected is the initial weight image corresponding to the to-be-fused image with the highest exposure in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure on the initial weight image to be corrected.

[0063] In the embodiment of the present application, the white object includes but is not limited to an object with a pure white color, or an object with a color close to white such as ivory white, silver white, snow white, pale white, gray white, greenish white, and moon white. For example, the white object is a white wall, white clothes, a white animal, or other white objects.

[0064] In the embodiment of the present application, the white object has a relatively high brightness in a real scene, and has the following characteristics in an image: low contrast, low saturation, and high gray value.

[0065] In the embodiment of the present application, when the to-be-corrected region in the initial weight image to be corrected is determined, a target region in which the white object is located in the to-be-fused image with the highest exposure can be determined first, and then the to-be-corrected region in the initial weight image to be corrected is determined according to a position of the target region in the to-be-fused image with the highest exposure, wherein the position of the to-be-corrected region in the initial weight image to be corrected is the same as the position of the target region in the to-be-fused image with the highest exposure; and the to-be-corrected region is a weight region of the white object in the initial weight image to be corrected.

[0066] In the embodiment of the present application, the target region in which the white object is located in the to-be-fused image with the highest exposure can be determined in various ways, for example, an object detection algorithm is used to detect the target region, or a cumulative histogram is used to detect the target region.

[0067] In this embodiment, considering that the target area where the white object is located in the image to be fused with the highest exposure has a high brightness and a high exposure value, which is far from the preset good exposure value, the brightness value assigned when allocating brightness based on the idea of ​​good exposure is relatively small, while other images to be fused that are close to the preset good exposure value are assigned a larger brightness value, which is contrary to the actual situation, causing the target area to be darkened during fusion. Therefore, it is necessary to detect the target area where the white object is located and increase the fusion weight value of the target area.

[0068] In step 304, the weight values ​​of each pixel in the region to be corrected are increased, while the weight values ​​of each pixel in other regions outside the region to be corrected in the initial weight image to be corrected remain unchanged, thus obtaining the target weight image.

[0069] In this embodiment of the application, in order to ensure that the fusion effect is consistent with the brightness of the real scene, when performing weighted image correction, the weight value of each pixel in the region to be corrected is increased, while the weight value of each pixel in other regions outside the region to be corrected in the initial weighted image to be corrected remains unchanged.

[0070] In some embodiments of this application, in order to reduce the amount of computation and improve the computation speed, the fusion weight value of the white object can be uniformly multiplied by a coefficient greater than 1. Accordingly, step 304 above may include the following steps: multiplying the weight value of each pixel in the area to be corrected by a target coefficient, wherein the target coefficient is a value greater than 1. For example, the target coefficient is α, 1 < α ≤ 1.2.

[0071] In some embodiments of this application, the weight values ​​of different pixels in the region to be corrected can be multiplied by different correction coefficients.

[0072] In one example, such as Figure 4 As shown, the initial weight image corresponding to the image to be fused with the highest exposure is W1, and the region to be corrected corresponding to the target region is Q1. Figure 4 As can be seen, the fusion weight value of Q1 is relatively small. In order to avoid the target area being darkened during fusion, the fusion weight value of Q1 is increased to obtain a new weight image, namely the target weight image W4.

[0073] In step 305, based on the target weight image and the initial weight images of the other images to be fused (excluding the image with the highest exposure in the target image sequence), the image with the highest exposure and the other images to be fused are fused to obtain the target image.

[0074] In some embodiments of this application, in order to improve the fusion speed, fusion can be performed based on the weighted image of the original size and the image to be fused.

[0075] In some embodiments of the present application, in order to ensure that the picture transition in the fusion result is natural, a pyramid strategy can be adopted to perform multi-scale decomposition on the to-be-fused images and the weight images, and fusion based on the multi-scale to-be-fused images and the weight images. Accordingly, the step 305 can include the following steps: step 3051 and step 3052.

[0076] In the step 3051, Laplacian pyramid decomposition is performed on the to-be-fused image with the highest exposure and the other to-be-fused images, and Gaussian pyramid decomposition is performed on the target weight image and the initial weight images of the to-be-fused images other than the to-be-fused image with the highest exposure in the target image sequence.

[0077] In the step 3052, the to-be-fused images of the corresponding scales obtained through the Laplacian pyramid decomposition are fused according to the weight images of the scales obtained through the Gaussian pyramid decomposition, to obtain the target image.

[0078] As can be seen from the above embodiments, in the embodiments, a target image sequence is obtained, wherein the target image sequence includes a plurality of to-be-fused images with different exposures under the same scene; an initial weight image of each to-be-fused image is determined according to an index parameter of a pixel point of each to-be-fused image, wherein the index parameter includes at least one of the following: contrast, saturation, and brightness; a to-be-corrected region in the initial weight image is determined, wherein the initial weight image to be corrected is the initial weight image corresponding to the to-be-fused image with the highest exposure in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure on the initial weight image to be corrected; a weight value of each pixel point in the to-be-corrected region is improved, and a weight value of each pixel point in other regions outside the to-be-corrected region in the initial weight image to be corrected is kept unchanged, to obtain a target weight image; and the to-be-fused image with the highest exposure and the other to-be-fused images are fused according to the target weight image and the initial weight images of the to-be-fused images other than the to-be-fused image with the highest exposure in the target image sequence, to obtain a target image.

[0079] As can be seen, in the embodiments of the present application, the weight value of the white object in the to-be-fused image with the highest exposure during fusion can be improved. Since the brightness of the white object in the to-be-fused image with the highest exposure is the highest, improving the weight value of the white object in the to-be-fused image with the highest exposure during fusion can correct the darkening of the non-overexposure region where the white object is located caused by multi-exposure fusion, and improve the image quality of the multi-exposure fusion.

[0080] In another embodiment provided by the present application, as shown in Figure 5 the step 303 can include the following steps: step 3031, step 3032, and step 3033.

[0081] In step 3031, the sum of the contrasts of the pixel points at the same position in each to-be-fused image of the target image sequence is calculated to obtain the sum of the contrasts of the pixel points at different positions.

[0082] In the embodiments of the present application, the sum of the contrasts of the pixel points at the same position in each to-be-fused image can be calculated to obtain a matrix with the same number of pixel points as the to-be-fused images, wherein one element in the matrix corresponds to one pixel position in the to-be-fused images, and the value of the element is the sum of the contrasts of the pixel points of the pixel position in each to-be-fused image.

[0083] In step 3032, a well-exposed region in the to-be-fused image with the highest exposure of the target image sequence is determined according to the sum of the contrasts of the pixel points at different positions, wherein the sum of the contrasts of the pixel points at each position in the well-exposed region is greater than a first threshold.

[0084] In the embodiments of the present application, it is considered that there is a positive correlation between the picture details and the sum of the contrasts, the sum of the contrasts is lower when the picture details are less, the sum of the contrasts is greater when the picture details are more, and the picture details are less when the image is overexposed, and the white objects that are darkened are usually in the non-overexposed region (i.e., the well-exposed region) of the image, so the well-exposed region can be accurately selected from the to-be-fused image with the highest exposure according to the sum of the contrasts of the target image sequence.

[0085] In step 3033, a target cumulative histogram corresponding to the well-exposed region is calculated, and a target region where the white object is located in the well-exposed region is determined according to the target cumulative histogram, and a position region corresponding to the target region in the initial weight image to be corrected is determined as the to-be-corrected region.

[0086] In the embodiments of the present application, it is considered that the gradient of the white object that is darkened on the cumulative histogram is small, so the target region where the white object is located can be selected from the well-exposed region according to the target cumulative histogram corresponding to the well-exposed region and the gradient performance of the white object on the cumulative histogram.

[0087] In some embodiments of the present application, the target cumulative histogram can be directly calculated on the well-exposed region.

[0088] In some embodiments of the present application, in order to ensure that the information in the target cumulative histogram is more comprehensive, the above step 3033 can include the following steps: performing inflation processing on the well-exposed region to obtain an inflated image region; and performing histogram calculation on the inflated image region to obtain the target cumulative histogram.

[0089] In some embodiments of the present application, considering that there is a negative correlation between the picture details of an image and the gradient of the cumulative histogram, the picture details are less when the gradient of the cumulative histogram is larger, and the picture details are more when the gradient of the cumulative histogram is smaller, the picture details of overexposed regions and underexposed regions in the image are less, the picture details of well-exposed regions are more, the white objects that are darkened are usually in the well-exposed regions of the image, and the contrast of the white objects in the image is low and the saturation of the white objects in the image is low, based on the above characteristics, in order to ensure the accuracy of the target region determination result, correspondingly, the step 3033 can include the following step: determining a region in which a pixel point with a cumulative histogram gradient value less than a second threshold value, a saturation less than a third threshold value, and a contrast less than a fourth threshold value in the well-exposed region as a target region in which a white object is located.

[0090] In the embodiments of the present application, the values of the first threshold value, the second threshold value, the third threshold value, and the fourth threshold value can be set according to actual conditions.

[0091] For ease of understanding, the determination process of the target region in which a white object is located in the embodiments of the present application will be explained in conjunction with the example shown in Figure 6

[0092] As shown in Figure 6 , the low-exposure image has many saturation pixels close to zero, so the cumulative histogram thereof increases sharply at the beginning. When a pixel is in a range in which the cumulative histogram changes slowly, it can be considered that the pixel is in a well-exposed region, that is, the pixel value of the region is relatively small, so the pixel value around the pixel changes greatly. In the medium-exposure image and the high-exposure image, the gradient of the cumulative histogram at a low pixel value is smaller than that of the low-exposure image, which also means that the pixel with a low value is located in a well-exposed or high-change region, so when the gradient of a pixel in the range of the cumulative histogram is smaller, a larger fusion weight value needs to be assigned to the pixel.

[0093] It can be seen that in the embodiments of the present application, the target region can be determined according to the characteristics of the darkened white object on the original image and the characteristics of the cumulative histogram, and then the to-be-corrected region in the weight image is determined, and the determination result is relatively accurate and the calculation amount is relatively small.

[0094] Figure 7 is a structural schematic diagram of an image fusion device provided by the embodiments of the present application, as shown in Figure 7 The image fusion device 700 can include an acquisition module 701, a first determination module 702, a second determination module 703, a correction module 704, and a fusion module 705.

[0095] The acquisition module 701 is configured to acquire a target image sequence, wherein the target image sequence includes a plurality of to-be-fused images with different exposure degrees in the same scene.​

[0096] The first determining module 702 is configured to determine an initial weight image of each of the to-be-fused images according to an index parameter of each pixel point of the to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation, and brightness.

[0097] The second determining module 703 is configured to determine a to-be-corrected region in a to-be-corrected initial weight image, wherein the to-be-corrected initial weight image is an initial weight image corresponding to a to-be-fused image with the highest exposure in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure on the to-be-corrected initial weight image.

[0098] The correcting module 704 is configured to increase a weight value of each pixel point in the to-be-corrected region and keep a weight value of each pixel point in other regions outside the to-be-corrected region in the to-be-corrected initial weight image unchanged, to obtain a target weight image.

[0099] The fusing module 705 is configured to fuse the to-be-fused image with the highest exposure and other to-be-fused images in the target image sequence according to the target weight image and initial weight images of the other to-be-fused images, to obtain a target image.

[0100] As can be seen from the above embodiment, in the embodiment, a target image sequence is obtained, wherein the target image sequence includes a plurality of to-be-fused images with different exposures under the same scene; an initial weight image of each to-be-fused image is determined according to an index parameter of each pixel point of the to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation, and brightness; a to-be-corrected region in a to-be-corrected initial weight image is determined, wherein the to-be-corrected initial weight image is an initial weight image corresponding to a to-be-fused image with the highest exposure in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure on the to-be-corrected initial weight image; a weight value of each pixel point in the to-be-corrected region is increased and a weight value of each pixel point in other regions outside the to-be-corrected region in the to-be-corrected initial weight image is kept unchanged, to obtain a target weight image; and the to-be-fused image with the highest exposure and other to-be-fused images are fused according to the target weight image and initial weight images of the other to-be-fused images, to obtain a target image.

[0101] It can be seen that in the embodiment of the present application, the weight value of the white object in the to-be-fused image with the highest exposure degree in fusion can be improved. Since the brightness of the white object in the to-be-fused image with the highest exposure degree is the highest, improving the weight value of the white object in the to-be-fused image with the highest exposure degree in fusion can correct the darkening of the non-overexposure region where the white object is caused by multi-exposure fusion, and improve the image quality of multi-exposure fusion.

[0102] Optionally, as an embodiment, the second determining module 703 can include:

[0103] The first calculating sub-module is configured to calculate the sum of the contrasts of the pixel points at the same position in each to-be-fused image of the target image sequence to obtain the sum of the contrasts of the pixel points at different positions;

[0104] The first determining sub-module is configured to determine a well-exposed region in the to-be-fused image with the highest exposure degree of the target image sequence according to the sum of the contrasts of the pixel points at different positions, wherein the sum of the contrasts of the pixel points at each position in the well-exposed region is greater than a first threshold value;

[0105] The second calculating sub-module is configured to calculate a target cumulative histogram corresponding to the well-exposed region;

[0106] The second determining sub-module is configured to determine a target region where a white object is located in the well-exposed region according to the target cumulative histogram;

[0107] The third determining sub-module is configured to determine a position region corresponding to the target region in the initial weight image to be corrected as a to-be-corrected region.

[0108] Optionally, as an embodiment, the second determining sub-module can include:

[0109] The determining unit is configured to determine a region where a pixel point with a cumulative histogram gradient value less than a second threshold value, a saturation less than a third threshold value, and a contrast less than a fourth threshold value is located in the well-exposed region as a target region where a white object is located.

[0110] Optionally, as an embodiment, the second calculating sub-module can include:

[0111] The preprocessing unit is configured to perform dilation processing on the well-exposed region to obtain a dilated image region;

[0112] The calculating unit is configured to perform histogram calculation on the dilated image region to obtain a target cumulative histogram.

[0113] Optionally, as an embodiment, the correction module 704 can include:

[0114] The correction sub-module is configured to multiply the weight value of each pixel in the to-be-corrected region by a target coefficient, wherein the target coefficient is a value greater than 1.

[0115] Optionally, as an embodiment, the fusion module 705 can include:

[0116] The decomposition sub-module is configured to perform Laplacian pyramid decomposition on the to-be-fused image with the highest exposure and the other to-be-fused images, and perform Gaussian pyramid decomposition on the initial weight image of the other to-be-fused images in the target weight image and the target image sequence except the to-be-fused image with the highest exposure.

[0117] The fusion sub-module is configured to fuse the to-be-fused images of corresponding scales obtained by the Laplacian pyramid decomposition according to the weight images of each scale obtained by the Gaussian pyramid decomposition, to obtain a target image.

[0118] Optionally, as an embodiment, the index parameter further includes a quality metric, wherein the quality metric is positively correlated with the gray value of a pixel, and the quality metric is used to balance the brightness of an image in the image fusion process.

[0119] Any one of the steps in the embodiments of the image fusion method provided in the present application and the specific operations in any one of the steps can be completed by the corresponding modules in the image fusion device. The processes of the corresponding operations completed by each module in the image fusion device are described with reference to the processes of the corresponding operations described in the embodiments of the image fusion method.

[0120] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts are described with reference to the parts of the method embodiments.

[0121] Figure 8 is a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832, for storing instructions, such as an application program, executable by the processing component 822. The application program stored in the memory 832 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute the instructions to perform the above method.

[0122] The electronic device can further include a power supply component 826 configured to perform power management of the electronic device, a wired or wireless network interface 850 configured to connect the electronic device to a network, and an input / output (I / O) interface 858. The electronic device can operate based on an operating system stored in the memory 832, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0123] According to still another embodiment of the present application, the present application further provides a computer readable storage medium, having stored thereon computer programs / instructions, which, when executed by a processor, implement the steps of the image fusion method according to any one of the above embodiments.

[0124] According to still another embodiment of the present application, the present application further provides a computer program product, comprising computer programs / instructions, which, when executed by a processor, implement the steps of the image fusion method according to any one of the above embodiments.

[0125] The embodiments in the present specification are described in progressive manner, with each embodiment focusing on the differences from other embodiments, and the same or similar parts among the embodiments can be mutually referred to.

[0126] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0127] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks can be implemented by computer program instructions. These computer program instructions can be provided to a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The apparatus that realizes the functions specified in a flow or multiple flows and / or blocks

[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0129] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and changes can be made thereto without departing from the scope of the present application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0130] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and do not imply singular or plural. Moreover, the terms "include", "have", or any other variant thereof are intended to encompass non-exclusive inclusions, such that processes, methods, articles, or apparatuses that comprise a set of elements not expressly listed are also within the scope of the present application. Unless otherwise specified, the use of the negative "does not" does not preclude the presence or addition of one or more elements indicated as being excluded.

[0131] The above provides a kind of image fusion method, electronic equipment and storage medium provided by the present application, have carried out detailed introduction in this paper, the principle and implementation mode of the present application are described in this paper by specific example, the above example is only for helping to understand the method of the present application and its core idea;For the general technical personnel of the field, according to the idea of the present application, there will be changes in specific implementation mode and application range, as described above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. An image fusion method, characterized by, The method comprises: obtaining a target image sequence, wherein the target image sequence comprises a plurality of to-be-fused images with different exposure degrees in the same scene; determining an initial weight image of each to-be-fused image according to an index parameter of a pixel point of each to-be-fused image, wherein the index parameter comprises at least one of contrast, saturation, and brightness; determining a to-be-corrected region in a to-be-corrected initial weight image, wherein the to-be-corrected initial weight image is an initial weight image corresponding to a to-be-fused image with the highest exposure degree in the target image sequence, and the to-be-corrected region is a region corresponding to a position of a white object in the to-be-fused image with the highest exposure degree on the to-be-corrected initial weight image; increasing weight values of pixel points in the to-be-corrected region and keeping weight values of pixel points in other regions outside the to-be-corrected region in the to-be-corrected initial weight image unchanged to obtain a target weight image; fusing the to-be-fused image with the highest exposure degree and other to-be-fused images in the target image sequence according to the target weight image and initial weight images of the other to-be-fused images to obtain a target image.

2. The method of claim 1, wherein, The method comprises: calculating a sum of contrasts of pixel points at the same position in each to-be-fused image of the target image sequence to obtain a sum of contrasts of pixel points at different positions; determining a good exposure region in the to-be-fused image with the highest exposure degree in the target image sequence according to the sum of contrasts of pixel points at different positions, wherein a sum of contrasts of pixel points at each position in the good exposure region is greater than a first threshold value; calculating a target cumulative histogram corresponding to the good exposure region, and determining a target region in which a white object is located in the good exposure region according to the target cumulative histogram; determining a position region corresponding to the target region in the to-be-corrected initial weight image as the to-be-corrected region.

3. The method of claim 2, wherein, The method comprises: determining a region in which pixel points with a cumulative histogram gradient value less than a second threshold value, a saturation less than a third threshold value, and a contrast less than a fourth threshold value in the good exposure region are located as the target region in which the white object is located.

4. The method of claim 2, wherein, The method comprises: performing inflation processing on the good exposure region to obtain an inflated image region; performing histogram calculation on the inflated image region to obtain a target cumulative histogram.

5. The method of claim 1, wherein, The method comprises: multiplying weight values of pixel points in the to-be-corrected region by a target coefficient, wherein the target coefficient is a value greater than 1.

6. The method of claim 1, wherein, The method comprises: performing Laplacian pyramid decomposition on the to-be-fused image with the highest exposure and the other to-be-fused images, and performing Gaussian pyramid decomposition on the target weight image and the initial weight images of the other to-be-fused images except the to-be-fused image with the highest exposure in the target image sequence; fusing the to-be-fused images of corresponding scales obtained by the Laplacian pyramid decomposition according to the weight images of the scales obtained by the Gaussian pyramid decomposition, to obtain a target image.

7. The method of claim 1, wherein, The index parameter further includes a quality metric, wherein the quality metric is positively correlated with a gray value of a pixel point, and the quality metric is used to balance brightness of an image in an image fusion process.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the method of any one of claims 1-7.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method of any one of claims 1-7.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method of any one of claims 1-7.

Images