Multi-exposure high-dynamic image synthesis method based on known exposure ratio

Abstract

The invention relates to the technical field of multi-exposure high-dynamic image synthesis, and particularly provides a multi-exposure high-dynamic image synthesis method based on a known exposure ratio, which comprises the following steps: for a group of image sequences with different exposure durations, establishing image mapping ways with different exposure durations based on Gamma transformation and an image exposure ratio, and mapping each image to the same exposure reference; a proper exposure area is enhanced by optimizing weight distribution, overexposure and underexposure areas are inhibited, an initial high-dynamic composite image is obtained through fusion, secondary fusion is further carried out on a reference image and the initial high-dynamic composite image, and the overall brightness and local details of the image are optimized. According to the method, detailed exposure nodes of each image do not need to be obtained, linear mapping is carried out according to the exposure ratio, the calculation complexity is greatly reduced, image detail processing is also ensured, and the method can be applied to scenes with high synthesis efficiency requirements, such as real-time on-orbit image processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-exposure high-dynamic image synthesis, and in particular relates to a multi-exposure high-dynamic image synthesis method based on a known exposure ratio. Background Art

[0002] Currently, the commonly used methods for high-dynamic synthesis of multiple images are mostly based on image fusion theory. They directly establish weight functions through multiple images and use multi-scale space to fuse images to improve the dynamic range of images. This method can achieve certain results. However, for practical on-orbit images, this type of algorithm is highly complex and often requires accurate knowledge of the exposure time of each image in order to fully utilize the relationship between illumination, exposure time and image grayscale values ​​for fusion from the perspective of physical models. Existing methods have problems such as high computational complexity, loss of details and over-fusion, and are not suitable for actual on-orbit image processing. Summary of the invention

[0003] In view of this, the present invention aims to provide a multi-exposure high-dynamic image synthesis method based on a known exposure ratio, which is suitable for dynamic range enhancement, real-time image processing and hardware embedded systems, especially for the real-time synthesis requirements of high-dynamic images in on-orbit imaging equipment. Through linear mapping and adaptive weight allocation, the amount of calculation is significantly reduced, while ensuring the naturalness and detail integrity of the synthesized image, avoiding the disadvantages of existing methods such as high computational complexity, detail loss and over-fusion.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows: The present invention provides a multi-exposure high-dynamic image synthesis method based on a known exposure ratio, comprising: establishing an image mapping relationship with different exposure times based on Gamma transformation, selecting an image in an image sequence as a reference image, and mapping all images in the image sequence to the reference image according to the image mapping relationship. The exposure time is: ; in, Indicates The image is mapped to the image after the reference image is exposed for a certain period of time. Indicates the image sequence images, Indicates the image sequence The exposure ratio of the image, represents the exposure ratio of the reference image, Represents the Gamma transformation factor; The mapped images are fused according to the following formula to obtain the initial high dynamic synthetic image: for: ; in, N represents the total number of images in the image sequence, p Represents any pixel in the image. represents the fusion weight operator; The initial high dynamic synthetic image is fused with the contrast-transformed reference image to obtain the final high dynamic synthetic image.

[0005] Preferably, the reference image is selected in the following manner: The image with the middle exposure time in all image sequences is selected as the reference image.

[0006] Preferably, by fusion weight operator Constrain overexposed and underexposed areas, and fuse weight operators The calculation formula is: ; in, Indicates The image p pixels, It is a grayscale adjustment parameter used to control the influence of the fusion weight operator on different exposure areas. The specific value is adjusted according to the actual application. Represents the middle value of the image's grayscale range.

[0007] Preferably, the final high dynamic synthetic image H for: ; in, represents the weight factor, represents the matrix dot product, represents the reference image after contrast transformation, .

[0008] Preferably, when the image is 8 bits, The value is 128; when the image is 16 bits, The value is 32768.

[0009] Compared with the prior art, the invention can achieve the following beneficial effects: The present invention performs linear mapping based on known exposure ratios, and combines gamma correction with adaptive weight allocation to perform efficient and high-quality HDR synthesis of image sequences. By fusing multiple images with different exposure times, the dynamic range of the scene is effectively expanded, and the bright and dark details are retained to the greatest extent by optimizing the allocation of fusion weights. Compared with the traditional method of obtaining the detailed exposure moment of each frame of the image, the fusion model of the present invention is simpler and has lower computational complexity. Problems such as loss of details and over-fusion will not occur. While ensuring the synthesis quality, the processing speed is significantly improved, and it can be applied to real-time high dynamic imaging fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 is a flowchart of a multi-exposure high dynamic image synthesis method based on a known exposure ratio provided in an embodiment of the present invention; Figure 2 is a first original image with different exposure times provided by an embodiment of the present invention; Figure 3 is a second original image with different exposure times provided by an embodiment of the present invention; Figure 4 It is a high dynamic image that is finally synthesized and outputted according to an embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical scheme and advantages of the invention clearer, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and do not constitute a limitation to the invention. Similar components in different embodiments use associated similar component numbers. In the following embodiments, many detailed descriptions are to enable the invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other components, materials, and methods. In some cases, some operations related to the invention are not shown or described in the specification, in order to avoid the core part of the invention being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0012] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a necessary sequence, unless otherwise specified that a certain sequence must be followed.

[0013] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0014] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0015] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0016] See also Figure 1 In one embodiment of the present invention, a multi-exposure high dynamic image synthesis method based on a known exposure ratio is provided. The core is to use the known exposure ratio for linear mapping, combined with Gamma correction and adaptive weight fusion, to achieve efficient and high-quality HDR synthesis to meet the requirements of real-time on-track image processing. The specific synthesis process is as follows: S1: Input N image sequences with different exposure times , the exposure time corresponding to each image in the image sequence is Indicates that i =1, 2, …, N The known exposure ratio of each image is expressed as: , i =1, 2, …, N In the embodiment of the present invention, only two images are used as examples to describe the synthesis process. Figure 2 and Figure 3 Two images with different exposure times, the exposure ratio of the two images is expressed as .

[0017] S2: According to the exposure ratio and Gamma transformation, the mapping relationship between different images is established. Assume and If they are two different images, the linear mapping relationship between them based on Gamma transformation is: ; in, and Indicates the exposure ratio of different images, Represents the Gamma transformation factor. Gamma transformation is used to enhance the contrast of an image to make it closer to the difference in human visual perception.

[0018] S3: Arrange all images in the image sequence according to the length of exposure time, select the middle value of exposure time as the benchmark, and its corresponding image as the reference image. The reference image is represented as , the exposure ratio of the reference image is expressed as The reference image should be as close as possible to human visual perception. When the reference image selected according to the above method obviously does not meet human visual perception, the reference image can be adjusted. Figure 2 and Figure 3 The fusion of two images is taken as an example, so it is considered that the image closer to human visual perception clarity is selected. Figure 3 The image shown is the reference image, ie, k=2.

[0019] S4: According to the mapping relationship between different images, the grayscale values ​​of all images in the image sequence are mapped to the exposure duration of the reference image, and the mapped images are expressed as follows after the unified exposure benchmark: ; in, Indicates The image is mapped to the image after the reference image exposure time, Gamma transformation factor is 0.5.

[0020] S5: In order to improve the naturalness of the fusion of images with different exposures after image fusion, it is necessary to reduce the weights of overexposed and underexposed areas while increasing the weights of moderately exposed areas. Establish a fusion weight operator for: ; in, Indicates The image p pixels, It is a grayscale adjustment parameter used to control the influence of the fusion weight operator on different exposure areas. The specific value is adjusted according to the actual application. Indicates the middle value of the grayscale range of the image. For example, if the image is 8 bits, The value is 128; when the image is 16 bits, The value is 32768.

[0021] S6: The mapped images in the weighted fusion image sequence are first fused according to the following normalization formula to obtain the initial high dynamic synthetic image for: ; in, N represents the total number of images in the image sequence, p Represents any pixel in the image. Represents the fusion weight operator.

[0022] S7: Further, the reference image after contrast transformation is combined with the initial high dynamic synthetic image obtained by initial fusion Perform secondary fusion to optimize detail retention and obtain Figure 4 The final high dynamic range composite image shown is: ; in, represents a weight factor. In the embodiment of the present invention, The value of is 0.5. represents the matrix dot product, represents the reference image after contrast transformation, .

[0023] The present invention constructs a mapping relationship between images through exposure ratio and Gamma transformation, maps images with different exposure times to the same exposure time reference, eliminates the need to obtain the detailed exposure time of each frame, significantly reduces the computational complexity, and enables on-orbit real-time image fusion.

[0024] In short, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.

[0025] The systems, devices, modules or units described in one or more of the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0026] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0027] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0028] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A multi-exposure high dynamic image synthesis method based on a known exposure ratio, characterized in that: include: Based on the Gamma transformation, an image mapping relationship with different exposure times is established, an image in the image sequence is selected as a reference image, and all images in the image sequence are mapped to the exposure time of the reference image according to the image mapping relationship: ； in, Indicates An image is mapped to the image after the reference image is exposed for a certain period of time, Indicates the image sequence images, Indicates the image sequence The exposure ratio of the image, represents the exposure ratio of the reference image, Represents the Gamma transformation factor; The mapped images are fused according to the following formula to obtain the initial high dynamic synthetic image: for: ； in, N represents the total number of images in the image sequence, p Represents any pixel in the image. represents the fusion weight operator; The initial high dynamic synthetic image is fused with the contrast-transformed reference image to obtain the final high dynamic synthetic image.

2. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 1, characterized in that: The reference image is selected in the following manner: An image with a middle exposure time in all image sequences is selected as the reference image.

3. The multi-exposure high dynamic image synthesis method based on a known exposure ratio according to claim 1, characterized in that: Through the fusion weight operator To constrain the overexposed and underexposed areas, the fusion weight operator The calculation formula is: ； in, Indicates The image p pixels, It is a grayscale adjustment parameter used to control the influence of the fusion weight operator on different exposure areas. The specific value is adjusted according to the actual application. Represents the middle value of the image's grayscale range.

4. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 1, characterized in that: The final high dynamic range composite image H for: ； in, represents the weight factor, represents the matrix dot product, represents the reference image after contrast transformation, .

5. The multi-exposure high dynamic image synthesis method based on known exposure ratio according to claim 3, characterized in that: When the image is 8 bits, The value is 128; when the image is 16 bits, The value is 32768.

Images