A multi-exposure high-dynamic-range image synthesis method based on known exposure ratios

Abstract

The present invention relates to the technical field of multi-exposure high-dynamic-range (HDR) image synthesis, and specifically provides a multi-exposure HDR image synthesis method based on known exposure ratios. For a set of image sequences with different exposure durations, an image mapping relationship for different exposure durations is established based on Gamma transformation and image exposure ratios, and each image is mapped to the same exposure reference. By optimizing the weight distribution, the moderately exposed region is enhanced, and the overexposed and underexposed regions are suppressed, and an initial HDR composite image is obtained through fusion. Then, the reference image is further fused with the initial HDR composite image to optimize the overall brightness and local details of the image. The present invention does not require obtaining the detailed exposure nodes of each image, and performs linear mapping according to the exposure ratio, greatly reducing the computational complexity, and also ensuring image detail processing. It can be applied to scenarios with high requirements for synthesis efficiency, such as real-time on-orbit image processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-exposure high-dynamic range image synthesis, and particularly relates to a multi-exposure high-dynamic range image synthesis method based on a known exposure ratio. Background Art

[0002] Currently, most of the commonly used methods for high-dynamic range synthesis based on multiple images are based on the theory of image fusion. A weight function is directly established through multiple images, and the images are fused using a multi-scale space to improve the dynamic range of the images. This method can achieve certain effects. However, for on-orbit images in practical applications, this type of algorithm has a high complexity. It is often necessary to accurately know the exposure time of each image in order to fully utilize the relationship between illuminance, exposure time, and image gray value for fusion from the perspective of the physical model. Existing methods have problems such as high computational complexity, detail loss, and over-fusion, and are not suitable for actual on-orbit image processing. Summary of the Invention

[0003] In view of this, the present invention aims to provide a multi-exposure high-dynamic range image synthesis method based on a known exposure ratio, which is applicable to dynamic range enhancement, real-time image processing, and hardware embedded systems. Especially for the real-time synthesis requirement of high-dynamic range images in on-orbit imaging devices, through linear mapping and adaptive weight allocation, the computational amount is significantly reduced, while ensuring the naturalness and detail integrity of the synthesized image, and avoiding the disadvantages of high computational complexity, detail loss, and over-fusion in existing methods.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] The present invention provides a multi-exposure high-dynamic range image synthesis method based on a known exposure ratio, including: establishing an image mapping relationship with different exposure durations based on Gamma transformation, selecting an image in the image sequence as a reference image, and mapping all images in the image sequence to the exposure duration of the reference image according to the image mapping relationship as:

[0006] ;

[0007] Wherein, represents the image after the th image is mapped to the exposure duration of the reference image, represents the th image in the image sequence, represents the exposure ratio of the th image in the image sequence, represents the exposure ratio of the reference image, represents the Gamma transformation factor;

[0008] The mapped images are fused according to the following formula to obtain the initial high-dynamic range composite image That is:

[0009] ;

[0010] Wherein, N represents the total number of images in the image sequence, p represents any pixel point in the image, represents the fusion weight operator;

[0011] The initial high-dynamic range composite image is fused with the reference image after contrast transformation to obtain the final high-dynamic range composite image.

[0012] Preferably, the reference image is selected as follows:

[0013] The image with the exposure duration as the median value in all the image sequences is selected as the reference image.

[0014] Preferably, the overexposed and underexposed regions are constrained by the fusion weight operator The calculation formula of the fusion weight operator is:

[0015] ;

[0016] Wherein, represents the th pixel point of the p th image, is a gray-scale adjustment parameter used to control the influence degree of the fusion weight operator on different exposure regions, and the specific value is adjusted according to the actual application, represents the median value of the image gray-scale range.

[0017] Preferably, the final high-dynamic range composite image H is:

[0018] ;

[0019] Wherein, represents the weight factor, represents the matrix dot product, represents the reference image after contrast transformation, .

[0020] Preferably, when the image is 8-bit, takes the value of 128; when the image is 16-bit, takes the value of 32768.

[0021] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0022] The present invention performs linear mapping based on a known exposure ratio, and combines Gamma correction and adaptive weight allocation to perform efficient and high-quality HDR synthesis on an image sequence. By fusing multiple images with different exposure durations, the dynamic range of the scene is effectively extended, and the details of the bright and dark parts are retained to the greatest extent by optimizing the allocation of fusion weights. Compared with the traditional method that requires obtaining the detailed exposure time of each frame image, the fusion model of the present invention is simpler, has a lower computational complexity, and does not have problems such as detail loss and over-fusion. While ensuring the synthesis quality, the processing speed is significantly improved, and it can be applied to real-time high-dynamic imaging fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0024] Figure 1 is a flowchart of a multi-exposure high-dynamic image synthesis method based on a known exposure ratio according to an embodiment of the present invention;

[0025] Figure 2 is the first original image with different exposure durations according to an embodiment of the present invention;

[0026] Figure 3 is the second original image with different exposure durations according to an embodiment of the present invention;

[0027] Figure 4 is the high-dynamic image of the final synthesized output according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention. Similar elements in different embodiments are labeled with related similar element numbers. In the following embodiments, many details are described to enable a better understanding of the present invention. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present invention are not shown or described in the specification to avoid the core part of the present invention being overwhelmed by excessive description. For those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the field.

[0029] It should be noted that, without conflict, the embodiments and the features in the embodiments in the present invention can be combined with each other to form various implementation manners. At the same time, the steps or actions in the method description can also be reordered or adjusted in an obvious manner by those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for clearly describing a certain embodiment, and do not mean that they are the necessary sequences, unless it is otherwise stated that a certain sequence must be followed.

[0030] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0031] In the description of the present invention, it should be noted that, unless otherwise clearly defined and limited, the terms "mounted", "connected", "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection, an electrical connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0032] The present invention will be described in detail below with reference to the drawings and in combination with embodiments.

[0033] Please refer to Figure 1 , in an embodiment of the present invention, a method for synthesizing multi-exposure high-dynamic-range images based on a known exposure ratio is provided. The core lies in performing linear mapping using the known exposure ratio, combining Gamma correction and adaptive weight fusion to achieve efficient and high-quality HDR synthesis to meet real-time on-orbit image processing. The specific synthesis process is as follows:

[0034] S1: Input an image sequence of N images with different exposure durations , the exposure duration corresponding to each image in the image sequence is represented by , where i = 1, 2, …, N . The known exposure ratio of each image is expressed as: , i = 1, 2, …, N . In the embodiment of the present invention, only two images are taken as examples to describe the synthesis process. For details, please refer to Figure 2 and Figure 3 two images with different exposure durations. The exposure ratios of the two images are expressed as .

[0035] S2: According to the exposure ratio and Gamma transformation, establish the mapping relationship between different images. Assume that and are two different images. Then, the linear mapping relationship between them obtained based on Gamma transformation is:

[0036] ;

[0037] where, and represent the exposure ratios of different images, represents the Gamma transformation factor. Gamma transformation is used to enhance the contrast of the image to make it closer to the difference in human visual perception.

[0038] S3: Arrange all the images in the image sequence according to the length of the exposure time, select the median value of the exposure duration as the reference, and the corresponding image as the reference image, which is represented by . The exposure ratio of the reference image is represented by . The reference image should be as close as possible to human visual perception. When the reference image selected according to the above method obviously does not meet human visual perception, the reference image can be adjusted. In the embodiment of the present invention, only the fusion of Figure 2 and Figure 3 two images is taken as an example. Therefore, it is considered that the image shown in Figure 3 which is closer to the clarity of human visual perception is selected as the reference image, that is, k = 2.

[0039] S4: According to the mapping relationship between different images, map the gray values of all the images in the image sequence to the exposure duration of the reference image, and unify the exposure reference to obtain the mapped image, which is represented as:

[0040] ;

[0041] where, represents the The image after mapping an image to the exposure duration of the reference image, Gamma transformation factor is 0.5.

[0042] S5: To improve the naturalness of the fusion of differently exposed images after image fusion, while increasing the weight of the moderately exposed area, it is also necessary to reduce the weights of the overexposed and underexposed areas. Establish a fusion weight operator as:

[0043] ;

[0044] wherein, represents the th p pixel of the th image, is a gray-scale adjustment parameter used to control the influence degree of the fusion weight operator on different exposure areas, and the specific value is adjusted according to the actual application. represents the middle value of the image gray-scale range. When the image is 8-bit, takes the value of 128; when the image is 16-bit,

[0045] S6: By weighted-fusing the mapped images in the image sequence, the initial high-dynamic range composite image is obtained through primary fusion according to the following normalization formula as:

[0046] ;

[0047] wherein, N represents the total number of images in the image sequence, p represents any pixel in the image, represents the fusion weight operator.

[0048] S7: Further, the reference image after contrast transformation and the initial high-dynamic range composite image obtained through initial fusion are subjected to secondary fusion to optimize the retention of details, and the final high-dynamic range composite image shown in Figure 4 is:

[0049] ;

[0050] wherein, represents the weight factor. In the embodiments of the present invention, takes the value of 0.5, represents the matrix dot product, represents the reference image after contrast transformation, .

[0051] The present invention constructs a mapping relationship between images through the exposure ratio and Gamma transformation, maps images with different exposure durations to the same exposure duration benchmark, and does not need to obtain the detailed moments of exposure for each frame of image, significantly reducing the computational complexity and enabling on-orbit real-time image fusion.

[0052] In summary, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this specification shall be included within the protection scope of this specification.

[0053] The systems, devices, modules or units described in one or more of the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0054] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, commodity or device including the said element.

[0055] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0056] The specific embodiments of this specification are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in a different order from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A multi-exposure high dynamic image synthesis method based on a known exposure ratio, characterized in that: include: Based on the Gamma transformation, an image mapping relationship with different exposure times is established, an image in the image sequence is selected as a reference image, and all images in the image sequence are mapped to the exposure time of the reference image according to the image mapping relationship: ； in, Indicates An image is mapped to the image after the reference image is exposed for a certain period of time, Indicates the image sequence images, Indicates the image sequence The exposure ratio of the image, represents the exposure ratio of the reference image, Represents the Gamma transformation factor; The mapped images are fused according to the following formula to obtain the initial high dynamic synthetic image: for: ； in, N represents the total number of images in the image sequence, p Represents any pixel in the image. represents the fusion weight operator; The initial high dynamic synthetic image is fused with the contrast-transformed reference image to obtain the final high dynamic synthetic image.

2. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 1, characterized in that: The reference image is selected in the following manner: An image with a middle exposure time in all image sequences is selected as the reference image.

3. The multi-exposure high dynamic image synthesis method based on a known exposure ratio according to claim 1, characterized in that: Through the fusion weight operator To constrain the overexposed and underexposed areas, the fusion weight operator The calculation formula is: ； in, Indicates The image is mapped to the image after the exposure time of the reference image. p The pixel value of a pixel, It is a grayscale adjustment parameter used to control the influence of the fusion weight operator on different exposure areas. The specific value is adjusted according to the actual application. Represents the middle value of the image's grayscale range.

4. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 1, characterized in that: The final high dynamic range composite image H for: ； in, represents the weight factor, represents the matrix dot product, represents the reference image after contrast transformation, , Represents the reference image.

5. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 3, characterized in that: When the image is 8 bits, The value is 128; when the image is 16 bits, The value is 32768.

Images