Image synthesis method and device, computer device and storage medium

Abstract

The application relates to an image synthesis method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a group of images, including a first to-be-synthesized image with a low exposure level and at least two second to-be-synthesized images with a high exposure level; determining an overexposure area in a high-luminance image after fusing the at least two to-be-synthesized images; determining a non-moving area in the overexposure area of the high-luminance image by comparing the at least two second to-be-synthesized images; determining a to-be-synthesized area in the first to-be-synthesized image based on the non-moving area; replacing the non-moving area in the overexposure area of the high-luminance image with an image of the to-be-synthesized area; and obtaining a synthesized image. Compared with a traditional method of synthesizing multiple images based on a high-luminance image, the method replaces the non-moving area in the overexposure area with an image with a low exposure level during synthesis, and the moving area in the overexposure area is not replaced, thereby preventing ghosting in the synthesized image and improving image definition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image synthesis method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer technology, people can now capture images using mobile devices such as smartphones and cameras. HDR (High Dynamic Range) image capture has become mainstream. HDR images are obtained by synthesizing multiple frames. These multiple frames may contain one or more frames with moving objects. Currently, the common method for synthesizing multiple frames is to use the brightest frame as a reference. However, due to the presence of moving objects in the multiple frames, and the brightness differences between these moving objects and other images, direct synthesis can lead to ghosting in the final image, resulting in a decrease in image sharpness.

[0003] Therefore, current image synthesis methods suffer from low image clarity. Summary of the Invention

[0004] Therefore, it is necessary to provide an image synthesis method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the clarity of image synthesis in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides an image synthesis method, the method comprising:

[0006] A set of images to be composited for the same object is obtained; the set of images to be composited includes a first image to be composited at a first exposure level and at least two second images to be composited at a second exposure level; the first exposure level is lower than the second exposure level; the at least two second images to be composited contain images with different exposure durations, and one of the images has the same image structure as the first image to be composited;

[0007] By fusing the at least two frames of the second image to be synthesized, a high-brightness image is obtained, and the overexposed areas in the high-brightness image are determined.

[0008] Based on the comparison results of the at least two frames of the second image to be synthesized, the non-moving region in the overexposed area of ​​the high-brightness image is determined; there are no moving objects in the non-moving region.

[0009] Based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region, the region to be synthesized in the first image to be synthesized is determined;

[0010] The non-moving area in the overexposed region of the high-brightness image is replaced with the image of the region to be synthesized. Based on the replaced high-brightness image, a synthesized image of the same object is obtained.

[0011] In one embodiment, obtaining a set of images to be synthesized of the same object includes:

[0012] A first original image with a first exposure time for a target object is acquired. The first original image is processed based on a first exposure level and a second exposure level to obtain a first image to be synthesized with a first sensitivity and a first exposure time at the first exposure level, and a short exposure image with a first sensitivity and a first exposure time at the second exposure level.

[0013] Acquire at least one frame of a second original image for a second exposure duration for the target object; process the at least one frame of the second original image based on a second exposure level to obtain at least one long exposure image with a second sensitivity and a second exposure duration at the second exposure level; the first sensitivity is greater than the second sensitivity; the brightness of the short exposure image and the long exposure image are consistent;

[0014] The set of images to be synthesized is obtained based on the first image to be synthesized, the short-exposure image, and the at least one long-exposure image.

[0015] In one embodiment, fusing the at least two frames of the second image to be synthesized to obtain a high-brightness image includes:

[0016] The non-overexposed areas in the at least two frames of the second images to be synthesized are determined, and the moving areas in the non-overexposed areas are determined based on the comparison results of the at least two frames of the second images to be synthesized; the moving areas contain moving objects.

[0017] Replace the image of the moving region in the non-overexposed region with the image of the corresponding region in the short-exposure image;

[0018] The image of the non-moving region in the non-overexposed region is replaced with the image of the corresponding region in the long exposure image; there are no moving objects in the non-moving region of the non-overexposed region.

[0019] A fused high-brightness image is obtained based on the fusion result of the image of the moving region in the replaced non-overexposed region and the image of the non-moving region in the non-overexposed region.

[0020] In one embodiment, determining the non-moving region in the overexposed area of ​​the high-brightness image based on the comparison results of the at least two frames of the second image to be synthesized includes:

[0021] Obtain the pixel overlap of the overexposed areas in the at least two frames of the second image to be synthesized;

[0022] Based on the region where the pixel overlap between the overexposed region of the short exposure image and the overexposed region of the long exposure image is greater than a preset overlap threshold, the non-moving region in the overexposed region of the high brightness image is determined.

[0023] In one embodiment, determining the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region includes:

[0024] The first image to be synthesized is matched with the non-moving region in the overexposed region to obtain the non-moving region in the first image to be synthesized.

[0025] The region to be synthesized is obtained from the non-moving region in the first image to be synthesized.

[0026] In one embodiment, determining the overexposed areas in a high-brightness image includes:

[0027] The region with a brightness greater than a preset brightness threshold in the fused high-brightness image is obtained, thus identifying the overexposed region in the high-brightness image.

[0028] In one embodiment, obtaining a composite image of the same object based on the replaced high-brightness image includes:

[0029] The replaced high-brightness image is subjected to at least one of the following processes: sharpening, brightness adjustment, and smoothing and denoising, to obtain a composite image of the same object with an image clarity greater than a preset clarity threshold.

[0030] Secondly, this application provides an image synthesis apparatus, the apparatus comprising:

[0031] An acquisition module is used to acquire a set of images to be synthesized from the same object; the set of images to be synthesized includes a first image to be synthesized at a first exposure level and at least two second images to be synthesized at a second exposure level; the first exposure level is lower than the second exposure level; the at least two second images to be synthesized contain images with different exposure durations, and one of the images has the same image structure as the first image to be synthesized;

[0032] The fusion module is used to fuse the at least two frames of the second image to be synthesized to obtain a high-brightness image and to determine the overexposed areas in the high-brightness image;

[0033] The first determining module is used to determine, based on the comparison results of the at least two frames of the second image to be synthesized, a non-moving region in the overexposed area of ​​the high-brightness image; the non-moving region does not contain any moving objects.

[0034] The second determining module is used to determine the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region;

[0035] The compositing module is used to replace the non-moving area in the overexposed area of ​​the high-brightness image with the image of the area to be composited, and obtain the composite image of the same object based on the replaced high-brightness image.

[0036] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0037] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0038] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0039] The aforementioned image compositing method, apparatus, computer equipment, storage medium, and computer program product acquire a set of images of the same object, including a first image to be composited at a low exposure level and at least two second images to be composited at high exposure levels. They then fuse the at least two high-exposure images to be composited, determine the overexposed areas in the fused high-brightness image, and determine the non-moving areas within the overexposed areas of the high-brightness image based on a comparison of the at least two second images to be composited. Based on these non-moving areas, they determine the areas to be composited in the first image to be composited, replace the non-moving areas in the overexposed areas of the high-brightness image with the images of the areas to be composited, and obtain a composite image of the same object based on the replaced high-brightness image. Compared to the traditional method of compositing multiple images based on a single high-brightness image, this method replaces the non-moving areas of the overexposed areas with the low-exposure image during compositing, while leaving the moving areas of the overexposed areas unreplaced, thus preventing ghosting in the composite image and improving image clarity. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating an image synthesis method in one embodiment;

[0041] Figure 2This is a flowchart illustrating the steps for acquiring the image to be synthesized in one embodiment;

[0042] Figure 3 This is a flowchart illustrating the fusion steps in one embodiment;

[0043] Figure 4 This is a structural block diagram of an image synthesis apparatus in one embodiment;

[0044] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] In one embodiment, such as Figure 1 As shown, an image synthesis method is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through the interaction between the terminal and the server. The method includes the following steps:

[0047] Step S202: Obtain a set of images to be synthesized from the same object; the set of images to be synthesized includes a first image to be synthesized at a first exposure level and at least two second images to be synthesized at a second exposure level; the first exposure level is lower than the second exposure level; at least two second images to be synthesized contain images with different exposure durations, and one of the images has the same image structure as the first image to be synthesized.

[0048] The images to be synthesized can be images that require compositing. This compositing process can be HDR image synthesis, where the terminal acquires a set of images to be synthesized and combines them into an HDR image. The acquired set of images to be synthesized can be images of the same object. For example, the terminal can use an image acquisition device to take a single photograph of the same object, obtaining multiple images from this photograph, which can then be used as a set of images to be synthesized. Each set of images to be synthesized includes at least three frames, and these at least three frames contain images with different exposure levels. For example, the set of images to be synthesized may include one first image at a first exposure level and at least two second images at a second exposure level. The first exposure level is lower than the second exposure level, meaning the exposure level of the first image to be synthesized is lower than that of the second image to be synthesized, resulting in a lower brightness for the first image to be synthesized compared to the second image to be synthesized. Specifically, the first exposure level can be called ev-, and the second exposure level can be called ev0. The ev value is called the exposure value, which reflects the combination of aperture size and shutter speed. In order to obtain an accurate exposure that is suitable for the correct representation of the subject, it is necessary to combine the exposure value and the exposure compensation amount. The exposure compensation amount is expressed by +3, +2, +1, 0, -1, -2, -3, etc. "+" means to increase the exposure based on the exposure value determined by metering, and "-" means to decrease the exposure. The corresponding number is the level of exposure compensation. For example, ev- can represent a negative exposure level, and ev0 can represent the 0th exposure level.

[0049] The aforementioned set of images to be synthesized may also include images with different exposure durations. For example, a set of images to be synthesized may simultaneously include images with short exposure durations and images with long exposure durations. Specifically, the first image to be synthesized at the first exposure level may be a short-exposure image, and the second image to be synthesized at the second exposure level may contain both short-exposure and long-exposure images. Furthermore, one of the at least two frames of the second images to be synthesized has the same image structure as the first image to be synthesized. Specifically, it may be the short-exposure second image to be synthesized. The terminal can generate the first image to be synthesized and the short-exposure second image to be synthesized from the same source, that is, the first image to be synthesized and the short-exposure second image to be synthesized come from the same original image. The terminal obtains the first image to be synthesized and the short-exposure second image to be synthesized through different image processing procedures, so that the image content and image structure of the first image to be synthesized and the short-exposure second image to be synthesized are consistent.

[0050] Step S204: Fuse at least two frames of the second image to be synthesized to obtain a high-brightness image, and determine the overexposed areas in the high-brightness image.

[0051] After acquiring the aforementioned set of images to be synthesized, the terminal can first synthesize at least two frames of the second images to be synthesized. Specifically, the terminal can fuse at least two frames of the second images at exposure level ev0. Since the brightness of exposure level ev0 is greater than that of exposure level ev-, the result of fusing at least two frames of the second images to be synthesized can be a high-brightness image. The brightness of each frame of the at least two second images to be synthesized can be consistent. The fusion process can be performed on non-overexposed areas within the second images to be synthesized. For example, the terminal can use a short exposure image of the moving area in the non-overexposed area to keep the moving object as still as possible, and use a long exposure image of the non-moving area in the non-overexposed area to make the image quality as clear as possible.

[0052] After fusing at least two frames of the second image to be synthesized for non-overexposed areas, the terminal can process the overexposed areas in the second image to be synthesized accordingly. The terminal can first determine the overexposed areas in the high-brightness image. For example, in one embodiment, determining the overexposed areas in the high-brightness image includes: acquiring areas in the fused high-brightness image whose brightness is greater than a preset brightness threshold, thus obtaining the overexposed areas in the high-brightness image. In this embodiment, the terminal can determine whether an area is an overexposed area by the brightness of each area in the high-brightness image. For example, the terminal can compare the brightness of each pixel in the fused high-brightness image with a preset brightness threshold, thereby obtaining areas in the high-brightness image whose brightness is greater than the preset brightness threshold. The terminal can then use these areas with brightness greater than the preset brightness threshold as the overexposed areas in the high-brightness image.

[0053] Step S206: Based on the comparison results of at least two frames of the second image to be synthesized, determine the non-moving area in the overexposed region of the high-brightness image; there are no moving objects in the non-moving area.

[0054] The second image to be synthesized can be an image with higher brightness than the first image to be synthesized. Furthermore, the brightness of the at least two frames of the second image to be synthesized can be consistent. The terminal can then compare the at least two frames of the second image to be synthesized and, based on the comparison result, determine the non-moving region within the overexposed area of ​​the high-brightness image. The non-moving region represents the absence of moving objects in that region, i.e., the absence of motion blur. The at least two frames of the second image to be synthesized contain both short-exposure and long-exposure images. Since the brightness of the short-exposure image and the long-exposure image are consistent, and both images target the same object, the terminal can compare the short-exposure and long-exposure images. When the terminal detects an area where the images are inconsistent between the short-exposure and long-exposure images, it can consider that the area contains a moving object. The terminal can then define these areas containing moving objects as moving regions, and further define the areas where the images are consistent between the short-exposure and long-exposure images as non-moving regions, indicating the absence of moving objects in those areas. Thus, by mapping the comparison result back to the high-brightness image, the non-moving region within the overexposed area of ​​the high-brightness image can be obtained.

[0055] Step S208: Determine the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region.

[0056] The non-moving region in the overexposed area can be the non-moving region in the overexposed area of ​​the high-brightness image obtained by the terminal after comparing the short-exposure image and the long-exposure image in at least two frames of the second image to be synthesized. The first image to be synthesized and the short-exposure image are images obtained from the same original image through different processing; therefore, the image content and structure of the first image to be synthesized are consistent with the short-exposure image. Thus, the non-moving region in the overexposed area determined by the terminal based on the short-exposure image and the long-exposure image can be consistent with the non-moving region in the first image to be synthesized. The terminal can match the non-moving region in the overexposed area of ​​the high-brightness image with the first image to be synthesized, for example, based on the position of the non-moving region in the overexposed area in the high-brightness image. Based on this matching result, the terminal can determine the region to be synthesized in the first image to be synthesized. The image in the region to be synthesized can be an image used for image replacement.

[0057] Step S210: Replace the non-moving area in the overexposed area of ​​the high-brightness image with the image of the area to be synthesized, and obtain the synthesized image of the same object based on the replaced high-brightness image.

[0058] In this process, after the terminal determines the region to be synthesized in the first image to be synthesized, it can replace the non-moving region in the overexposed region of the high-brightness image with the image of the region to be synthesized. The first image to be synthesized can be an image with lower brightness, that is, an image without overexposure. After the terminal replaces the image in the non-moving region in the overexposed region of the high-brightness image with the image of the region to be synthesized, it can remove the overexposure of the replaced region and improve the image clarity.

[0059] After replacing the non-moving area in the overexposed region of the aforementioned high-brightness image with the image of the area to be synthesized, the terminal can obtain a synthesized image of the same object based on the replaced high-brightness image. The terminal can perform post-processing on the replaced high-brightness image to improve its transparency and clarity. For example, in one embodiment, obtaining the synthesized image of the same object based on the replaced high-brightness image includes: performing at least one of the following processes on the replaced high-brightness image: sharpening, brightness adjustment, and smoothing / denoising, to obtain a synthesized image of the same object with an image clarity greater than a preset clarity threshold. In this embodiment, the terminal can make corresponding adjustments to the replaced high-brightness image to improve its transparency and brightness. For example, the terminal can perform at least one of the following processes on the replaced high-brightness image: sharpening, brightness adjustment, and smoothing / denoising, thereby improving the transparency and clarity of the replaced high-brightness image. After the terminal makes the above adjustments to the high-brightness image after replacement, it can obtain a composite image based on the above set of images to be composited. Since the image of the corresponding area in the first image to be composited with the ev-exposure level is used to replace the non-moving area in the overexposed area, and the overexposed area in the moving area is maintained, it is ensured that the composite image will not produce ghosting due to the overlap of moving and stationary objects.

[0060] In the aforementioned image compositing method, a set of images of the same object is acquired, including a first image to be composited at a low exposure level and at least two second images to be composited at a high exposure level. The at least two high-exposure images are then fused to determine the overexposed areas in the fused high-brightness image. Based on the comparison results of the at least two second images to be composited, the non-moving areas within the overexposed areas of the high-brightness image are determined. Based on these non-moving areas, the composite area in the first image to be composited is determined. The non-moving areas within the overexposed areas of the high-brightness image are replaced with the images of the composite areas. Based on the replaced high-brightness image, a composite image of the same object is obtained. Compared to the traditional method of compositing multiple images based on a single high-brightness image, this method replaces the non-moving areas of the overexposed areas with images of the low exposure level during compositing, while leaving the moving areas of the overexposed areas unreplaced. This prevents ghosting in the composite image and improves image clarity.

[0061] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating the steps for acquiring images to be synthesized in one embodiment. The acquisition of a set of images to be synthesized for the same object includes: Step S302: Acquiring a first original image for a first exposure duration targeting the target object; processing the first original image based on a first exposure level and a second exposure level to obtain a first image to be synthesized at a first sensitivity and a first exposure duration at the first exposure level, and a short exposure image at a second sensitivity and a first exposure duration at the second exposure level; Step S304: Acquiring at least one frame of a second original image for a second exposure duration targeting the target object; processing at least one frame of the second original image based on the second exposure level to obtain at least one long exposure image at a second sensitivity and a second exposure duration at the second exposure level; the first sensitivity is greater than the second sensitivity; the brightness of the short exposure image and the long exposure image is consistent; Step S306: Obtaining a set of images to be synthesized based on the first image to be synthesized, the short exposure image, and at least one long exposure image.

[0062] In this embodiment, the terminal can acquire a set of images to be synthesized of the same object through an image acquisition device. For example, the terminal can capture multiple frames of images of the same object using a camera to obtain a set of images to be synthesized. Among the set of images to be synthesized acquired by the terminal, there may be multiple frames with different shooting parameters. The same object can be a target object. The terminal can acquire a first original image of the target object under a first exposure time, and at least one second original image of the target object under a second exposure time. The first exposure time is shorter than the second exposure time, and both the first and second original images can be raw format images, which contain camera sensor data information necessary to create a visual image. The terminal can perform corresponding processing on the original images to obtain the first and second images to be synthesized. Specifically, the processing can be ISP (Image Signal Processor) processing of the raw format image, which mainly performs post-processing on the signal output by the front-end image sensor. Through ISP processing, the terminal can better restore scene details under different optical conditions.

[0063] For the aforementioned first original image, the terminal can perform different ISP processing on the first original image to obtain multiple images to be composited at different exposure levels. The first original image has a corresponding first exposure duration and first ISO. The terminal can process the first original image based on the first exposure level, thereby obtaining a first composite image with the first ISO and first exposure duration at the first exposure level; the terminal can also process the first original image based on a second exposure level, thereby obtaining a short exposure image with the first ISO and first exposure duration at the second exposure level. That is, the terminal can obtain composite images at different exposure levels based on a single first original image through different processing methods. Specifically, the first composite image can be an image at an ev- exposure level, and the short exposure image can be an image at an ev0 exposure level. The exposure duration and ISO of the first composite image and the short exposure image can be the same. That is, the first composite image and the short exposure image can be generated from the same source, making the image content and image structure of the first composite image and the short exposure image consistent, facilitating the subsequent determination of the composite region.

[0064] For the second original image, the terminal can process at least one frame of the second original image based on the second exposure level to obtain at least one long exposure image with a second sensitivity and a second exposure duration at the second exposure level. Since the exposure duration of the second original image is relatively long, there may be moving areas due to object movement. To ensure consistent brightness between the short exposure image and the long exposure image, the terminal can set the first sensitivity to be greater than the second sensitivity. This allows the terminal to obtain a short exposure image based on a shorter exposure time and a higher first sensitivity, and a long exposure image based on a longer exposure time and a lower second sensitivity. This allows for subsequent comparison of the short exposure image and the long exposure image with consistent brightness to determine the moving areas. The at least one long exposure image can be an image with the same exposure level as the short exposure image, for example, both at ev0 exposure level. This allows the terminal to obtain a combination of short exposure images and long exposure images with the same brightness. Specifically, during image synthesis, the terminal can acquire one first image to be synthesized at ev- and N second images to be synthesized at ev0, where N≥2. The aforementioned N ev0 images may include images with short exposure times and high ISO, as well as images with long exposure times and low ISO, ensuring that the brightness of the N ev0 images to be synthesized is consistent. Furthermore, the first image to be synthesized (ev-) and the second image to be synthesized (ev0) with short exposure times have the same image content and structure. After acquiring the first image to be synthesized, the short exposure image, and the long exposure image, the terminal can obtain a set of images to be synthesized for the same object based on these images. This set of images to be synthesized contains at least three frames.

[0065] In this embodiment, the terminal can obtain multiple images with consistent content and structure at different exposure levels based on the same original image. Therefore, by identifying the moving areas in the high-brightness image to be synthesized, the terminal can correlate them with the moving areas in the low-brightness image, improving the accuracy of image synthesis. Furthermore, by generating short-exposure and long-exposure images with consistent brightness, and comparing these images, the terminal can determine the moving areas within the images, enabling image synthesis based on these moving areas and improving the clarity of the synthesized images.

[0066] In one embodiment, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating the fusion steps in one embodiment. The above-mentioned fusion of at least two frames of second images to be synthesized to obtain a high-brightness image includes: Step S402: Determining non-overexposed areas in at least two frames of second images to be synthesized, and determining moving areas within the non-overexposed areas based on the comparison results of the at least two frames of second images to be synthesized; the moving areas contain moving objects; Step S404: Replacing the image of the moving area in the non-overexposed areas with the image of the corresponding area in the short-exposure image; Step S406: Replacing the image of the non-moving area in the non-overexposed areas with the image of the corresponding area in the long-exposure image; the non-moving areas in the non-overexposed areas do not contain moving objects; Step S408: Obtaining the fused high-brightness image based on the fusion result of the replaced images of the moving areas in the non-overexposed areas and the images of the non-moving areas in the non-overexposed areas.

[0067] In this embodiment, the terminal can fuse at least two frames of the second images to be synthesized at the second exposure level to obtain a high-brightness image. For example, the terminal can first determine the non-overexposed areas in the at least two frames of the second images to be synthesized. Specifically, the terminal can detect the brightness of each pixel in each second image to be synthesized and identify areas with brightness lower than a preset brightness threshold as non-overexposed areas. The terminal can also determine the moving areas within the non-overexposed areas based on the comparison results of the at least two frames of the second images to be synthesized. A moving area indicates an area where a moving object exists, resulting in motion blur. Specifically, the terminal can compare the at least two frames of the second images to be synthesized to obtain pixel overlap, and identify areas in the non-overexposed areas where the pixel overlap is lower than a preset overlap threshold as moving areas within the non-overexposed areas. After the terminal determines the moving areas within the non-overexposed areas, the other areas within the non-overexposed areas can be identified as non-moving areas.

[0068] The aforementioned at least two frames of the second image to be synthesized include a short-exposure image and at least one long-exposure image. The terminal can use the short-exposure image as a reference frame, replacing the image of the moving area in the non-overexposed area with the image of the corresponding area in the short-exposure image, and replacing the image of the non-moving area in the non-overexposed area with the image of the corresponding area in the long-exposure image. Specifically, when using the short-exposure image at exposure level EV0 as the reference frame, for non-overexposed areas in the image, if there is a moving area, the terminal can use the image of the corresponding area in the short-exposure image at that location to ensure that the moving object is frozen and clearly presented; while for the remaining non-moving areas in the non-overexposed area, the image of the corresponding area in the long-exposure image can be used to ensure clear image quality. The terminal determines the replacement images for each area in the aforementioned second image to be synthesized, and can obtain the fused high-brightness image based on the fusion result of the images of the moving areas in the non-overexposed area and the images of the non-moving areas in the non-overexposed area.

[0069] Through this embodiment, the terminal can use the image of the corresponding area in the short exposure image in the moving area of ​​the non-overexposed area in the second image to be synthesized, and use the image of the corresponding area in the long exposure image in the non-moving area of ​​the non-overexposed area, thereby improving the clarity of the fused high-brightness image, and thus improving the clarity of the final synthesized image.

[0070] In one embodiment, determining the non-moving region in the overexposed area of ​​the high-brightness image based on the comparison results of at least two frames of the second images to be synthesized includes: obtaining the pixel overlap of the overexposed region images in at least two frames of the second images to be synthesized; and determining the non-moving region in the overexposed area of ​​the high-brightness image based on the region where the pixel overlap of the overexposed region images of the short-exposure image and the overexposed region images of the long-exposure image is greater than a preset overlap threshold.

[0071] In this embodiment, the terminal can determine the non-moving regions in the overexposed areas of the at least two frames of the second images to be synthesized by comparison. For example, the terminal can perform pixel comparison on the at least two frames of the second images to be synthesized and obtain the pixel overlap of the overexposed areas in the at least two frames of the second images to be synthesized. The terminal can obtain the pixel overlap of the overexposed areas in the short exposure image and the overexposed areas in the long exposure image within a region with a preset overlap threshold, and determine the non-moving regions in the overexposed areas of the high-brightness image based on these regions. Specifically, the at least two frames of the second images to be synthesized include a short exposure image and a long exposure image, and the brightness of the short exposure image and the long exposure image is consistent. Therefore, the overexposed areas determined based on the short exposure image and the long exposure image will be consistent with the overexposed areas of the fused high-brightness image. Therefore, the terminal can determine the non-moving regions in the overexposed areas by comparing the short exposure image and the long exposure image, while other regions in the overexposed areas are determined as moving regions in the overexposed areas. Here, a moving region indicates that there is a moving object in the region, and a non-moving region indicates that there is no moving object in the region.

[0072] In this embodiment, the terminal can determine the non-moving region of the overexposed area in a high-brightness image based on the pixel overlap. Therefore, the terminal can perform image replacement based on the non-moving region, improving the clarity of the synthesized image.

[0073] In one embodiment, determining the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region includes: matching the first image to be synthesized with the non-moving region in the overexposed region to obtain the non-moving region in the first image to be synthesized; and obtaining the region to be synthesized based on the non-moving region in the first image to be synthesized.

[0074] In this embodiment, the terminal can determine the areas to be synthesized in the first image to replace the non-moving areas in the overexposed regions by mapping the non-moving areas in the determined overexposed regions back to the first image to be synthesized. Since the first image to be synthesized and the short-exposure image originate from the same original image and have the same image structure and content, the non-moving areas in the overexposed regions determined by the terminal based on the short-exposure image can correspond to the first image to be synthesized. The terminal can match the first image to be synthesized with the non-moving areas in the determined overexposed regions to obtain the non-moving areas in the first image to be synthesized, thereby obtaining the area to be synthesized based on the non-moving areas in the first image to be synthesized. The image in the area to be synthesized can be used to replace the image of the non-moving areas in the overexposed regions of the high-brightness image. Specifically, since the moving and non-moving regions in the overexposed regions have been identified, when the terminal wants to use an EV-frame image in the overexposed regions of the high-brightness image, it can decide whether to use it based on whether the position belongs to a moving region, so that the EV-frame image can be accurately restored to the synthesized image and avoid ghosting.

[0075] Through this embodiment, the terminal can identify the moving and non-moving regions in the overexposed area of ​​a high-brightness image, and based on the correspondence between the short-exposure image and the first image to be synthesized, determine the region to be synthesized in the first image to be synthesized that corresponds to the position of the non-moving region in the overexposed area. Thus, the terminal can synthesize the image based on the image in the region to be synthesized, thereby improving the clarity of the synthesized image.

[0076] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0077] Based on the same inventive concept, this application also provides an image compositing apparatus for implementing the image compositing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image compositing apparatus embodiments provided below can be found in the limitations of the image compositing method described above, and will not be repeated here.

[0078] In one embodiment, such as Figure 4 As shown, an image synthesis apparatus is provided, comprising: an acquisition module 500, a fusion module 502, a first determination module 504, a second determination module 506, and a synthesis module 508, wherein:

[0079] The acquisition module 500 is used to acquire a set of images to be synthesized from the same object; the set of images to be synthesized includes a first image to be synthesized at a first exposure level and at least two second images to be synthesized at a second exposure level; the first exposure level is lower than the second exposure level; at least two second images to be synthesized contain images with different exposure durations, and one of the images has the same image structure as the first image to be synthesized.

[0080] The fusion module 502 is used to fuse at least two frames of the second image to be synthesized to obtain a high-brightness image and to determine the overexposed areas in the high-brightness image.

[0081] The first determining module 504 is used to determine the non-moving region in the overexposed area of ​​the high-brightness image based on the comparison results of at least two frames of the second image to be synthesized; there is no moving object in the non-moving region.

[0082] The second determining module 506 is used to determine the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region.

[0083] The compositing module 508 is used to replace the non-moving area in the overexposed area of ​​the high-brightness image with the image of the area to be composited, and obtain a composite image of the same object based on the replaced high-brightness image.

[0084] In one embodiment, the acquisition module 500 is specifically configured to acquire a first original image of a target object with a first exposure duration, process the first original image based on a first exposure level and a second exposure level respectively, to obtain a first image to be synthesized with a first sensitivity and a first exposure duration at the first exposure level, and a short exposure image with a first sensitivity and a first exposure duration at the second exposure level; acquire at least one frame of a second original image of the target object with a second exposure duration, process the at least one frame of the second original image based on the second exposure level, to obtain at least one frame of a long exposure image with a second sensitivity and a second exposure duration at the second exposure level; the first sensitivity is greater than the second sensitivity; the brightness of the short exposure image and the long exposure image is consistent; and obtain a set of images to be synthesized based on the first image to be synthesized, the short exposure image, and at least one frame of the long exposure image.

[0085] In one embodiment, the fusion module 502 is specifically configured to determine non-overexposed areas in at least two frames of the second images to be synthesized, and determine moving areas in the non-overexposed areas based on the comparison results of the at least two frames of the second images to be synthesized; the moving areas contain moving objects; replace the image of the moving areas in the non-overexposed areas with the image of the corresponding area in the short-exposure image; replace the image of the non-moving areas in the non-overexposed areas with the image of the corresponding area in the long-exposure image; the non-moving areas in the non-overexposed areas do not contain moving objects; and obtain a fused high-brightness image based on the fusion result of the replaced images of the moving areas in the non-overexposed areas and the images of the non-moving areas in the non-overexposed areas.

[0086] In one embodiment, the first determining module 504 is specifically used to obtain the pixel overlap of the overexposed areas in at least two frames of the second image to be synthesized; and to determine the non-moving area in the overexposed area of ​​the high-brightness image based on the area where the pixel overlap of the overexposed area of ​​the short exposure image and the overexposed area of ​​the long exposure image is greater than a preset overlap threshold.

[0087] In one embodiment, the second determining module 506 is specifically used to match the first image to be synthesized with the non-moving region in the overexposed region to obtain the non-moving region in the first image to be synthesized; and to obtain the region to be synthesized based on the non-moving region in the first image to be synthesized.

[0088] In one embodiment, the fusion module 502 is specifically used to obtain the region in the fused high-brightness image whose brightness is greater than a preset brightness threshold, thereby obtaining the overexposed region in the high-brightness image.

[0089] In one embodiment, the above-mentioned synthesis module 508 is specifically used to perform at least one of the following processes on the replaced high-brightness image: sharpening, brightness adjustment, and smoothing and denoising, to obtain a synthesized image of the same object with an image clarity greater than a preset clarity threshold.

[0090] Each module in the aforementioned image synthesis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0091] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image synthesis method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0092] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0093] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the image synthesis method described above.

[0094] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image synthesis method described above.

[0095] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the image synthesis method described above.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image synthesis method, characterized in that, The method includes: A set of images to be composited for the same object is obtained; the set of images to be composited includes one frame of a first image to be composited at a first exposure level and at least two frames of a second image to be composited at a second exposure level; the first exposure level is less than the second exposure level; the at least two frames of the second images to be composited contain images with different exposure durations, including short exposure images and long exposure images; and one of the short exposure images has the same image structure as the first image to be composited; the first image to be composited represents an image with a first ISO and a first exposure duration at the first exposure level; the short exposure image represents an image with a first ISO and a first exposure duration at the second exposure level, and the long exposure image represents an image with a second ISO and a second exposure duration at the second exposure level; the short exposure image and the long exposure image have the same brightness; the first exposure duration is less than the second exposure duration. A high-brightness image is obtained by fusing the at least two frames of the second image to be synthesized, and the overexposed areas in the high-brightness image are determined; the image of the moving area in the non-overexposed area of ​​the high-brightness image is the image of the corresponding area in the short-exposure image; the image of the non-moving area in the non-overexposed area is the image of the corresponding area in the long-exposure image; Based on the comparison results of the at least two frames of the second image to be synthesized, the non-moving region in the overexposed area of ​​the high-brightness image is determined; there are no moving objects in the non-moving region. Based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region, the region to be synthesized in the first image to be synthesized is determined; The non-moving regions in the overexposed areas of the high-brightness image are replaced with the image of the region to be synthesized, while the moving regions in the overexposed areas remain overexposed. Based on the replaced high-brightness image, a synthesized image of the same object is obtained.

2. The method according to claim 1, characterized in that, The process of obtaining a set of images to be synthesized that are identical to the original object includes: A first original image with a first exposure time for a target object is acquired. The first original image is processed based on a first exposure level and a second exposure level to obtain a first image to be synthesized with a first sensitivity and a first exposure time at the first exposure level, and a short exposure image with a first sensitivity and a first exposure time at the second exposure level. Acquire at least one frame of a second original image for a second exposure duration for the target object, process the at least one frame of the second original image based on a second exposure level, and obtain at least one long exposure image with a second sensitivity and a second exposure duration at the second exposure level; the first sensitivity is greater than the second sensitivity; The set of images to be synthesized is obtained based on the first image to be synthesized, the short-exposure image, and the at least one long-exposure image.

3. The method according to claim 2, characterized in that, The process of fusing the at least two frames of the second image to be synthesized to obtain a high-brightness image includes: Identify the non-overexposed areas in the at least two frames of the second image to be synthesized; Based on the comparison results of the at least two frames of the second image to be synthesized, a moving region is determined in the non-overexposed region; the moving region contains a moving object. Replace the image of the moving region in the non-overexposed region with the image of the corresponding region in the short-exposure image; The image of the non-moving region in the non-overexposed region is replaced with the image of the corresponding region in the long exposure image; there are no moving objects in the non-moving region of the non-overexposed region. A fused high-brightness image is obtained based on the fusion result of the image of the moving region in the replaced non-overexposed region and the image of the non-moving region in the non-overexposed region.

4. The method according to claim 2, characterized in that, The step of determining the non-moving region in the overexposed area of ​​the high-brightness image based on the comparison results of the at least two frames of the second image to be synthesized includes: Obtain the pixel overlap of the overexposed areas in the at least two frames of the second image to be synthesized; Based on the region where the pixel overlap between the overexposed region of the short exposure image and the overexposed region of the long exposure image is greater than a preset overlap threshold, the non-moving region in the overexposed region of the high brightness image is determined.

5. The method according to claim 1, characterized in that, The step of determining the region to be synthesized in the first image based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region includes: The first image to be synthesized is matched with the non-moving region in the overexposed region to obtain the non-moving region in the first image to be synthesized. The region to be synthesized is obtained from the non-moving region in the first image to be synthesized.

6. The method according to claim 1, characterized in that, The process of determining overexposed areas in a high-brightness image includes: The region with a brightness greater than a preset brightness threshold in the fused high-brightness image is obtained, thus identifying the overexposed region in the high-brightness image.

7. The method according to any one of claims 1 to 6, characterized in that, The step of obtaining the composite image of the same object based on the replaced high-brightness image includes: The replaced high-brightness image is subjected to at least one of the following processes: sharpening, brightness adjustment, and smoothing and denoising, to obtain a composite image of the same object with an image clarity greater than a preset clarity threshold.

8. An image synthesis apparatus, characterized in that, The device includes: An acquisition module is used to acquire a set of images to be composited of the same object; the set of images to be composited includes one frame of a first image to be composited at a first exposure level and at least two frames of a second image to be composited at a second exposure level; the first exposure level is less than the second exposure level; the at least two frames of the second images to be composited contain images with different exposure durations, including short exposure images and long exposure images; and one of the short exposure images has the same image structure as the first image to be composited; the first image to be composited represents an image with a first ISO and a first exposure duration at the first exposure level; the short exposure image represents an image with a first ISO and a first exposure duration at the second exposure level, and the long exposure image represents an image with a second ISO and a second exposure duration at the second exposure level; the short exposure image and the long exposure image have the same brightness; the first exposure duration is less than the second exposure duration. The fusion module is used to fuse the at least two frames of the second image to be synthesized to obtain a high-brightness image, and to determine the overexposed areas in the high-brightness image; the image of the moving area in the non-overexposed area of ​​the high-brightness image is the image of the corresponding area in the short-exposure image; the image of the non-moving area in the non-overexposed area is the image of the corresponding area in the long-exposure image; The first determining module is used to determine, based on the comparison results of the at least two frames of the second image to be synthesized, a non-moving region in the overexposed area of ​​the high-brightness image; the non-moving region does not contain any moving objects. The second determining module is used to determine the region to be synthesized in the first image to be synthesized based on the matching result between the first image to be synthesized and the non-moving region in the overexposed region; The compositing module is used to replace the non-moving areas in the overexposed areas of the high-brightness image with the image of the area to be composited, keep the moving areas in the overexposed areas in an overexposed state, and obtain the composited image of the same object based on the replaced high-brightness image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images