Method and electronic device for generating captured image

Abstract

A method comprises: capturing a first initial image within a first dynamic range, wherein the first initial image is captured under different imaging settings; generating an intermediate image of a second dynamic range based on the first initial image, wherein the second dynamic range is higher than the first dynamic range; detecting a light source area based on a light detection image to generate a light source map, wherein the light detection image is at least one image in the first initial image and is suitable for detecting the light source area; generating a captured image based on the intermediate image and the light source map, wherein the position of the light source area of ​​the captured image is determined based on the light source map, and the color of the light source area of ​​the captured image is determined based on the intermediate image.

Description

Technical Field

[0001] The present application relates to a method for generating a captured image in an electronic device including a camera assembly and the electronic device. Background Art

[0002] Electronic devices such as smart phones and tablet terminals are widely used in daily life. Nowadays, many electronic devices are equipped with camera components to capture images. Some electronic devices are portable and easy to carry, so that users can easily capture images anytime and anywhere using the camera components of the electronic devices.

[0003] However, if the user of the electronic device captures an image at night, the captured image includes one or more overexposed or haloed light source areas. After capturing the image, the overexposed light source areas and halo areas in the captured image can be subjected to image correction processing.

[0004] However, even if the captured image has been subjected to image correction processing, the quality of the light source area on the captured image is sometimes not enough for the user, for example, because its color is not suitable for the light source area or the bokeh effect introduced into the light source area on the captured image is too strong. Therefore, a technology for improving the quality of the light source area on the captured image is needed. Summary of the invention

[0005] The present application aims to solve at least one of the above technical problems. Therefore, the present application needs to provide a method and an electronic device for generating a captured image.

[0006] According to the present application, a method for generating a captured image in an electronic device including a camera assembly may include:

[0007] capturing, by the camera assembly, a plurality of first initial images in a first dynamic range, wherein the plurality of first initial images are captured under different imaging settings;

[0008] generating an intermediate image in a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

[0009] Detecting a light source area according to a light detection image to generate a light source map, wherein the light detection image is at least one image among the plurality of first initial images and is suitable for detecting the light source area;

[0010] A captured image is generated based on the intermediate image and the light source map, wherein the position of the light source area of ​​the captured image is determined based on the light source map, and the color of the light source area of ​​the captured image is determined based on the intermediate image.

[0011] In some embodiments, in the method, capturing the plurality of first initial images may include: capturing the plurality of first initial images at different exposure values.

[0012] In some embodiments, in the method, the light detection image may be one of the plurality of first initial images captured at a lowest exposure value.

[0013] In some embodiments, in the method, the minimum exposure value may be low enough to detect light from an area other than the light source area in the first initial image.

[0014] In some embodiments, in this method, the lowest exposure value may be a negative value.

[0015] In some embodiments, the method may further include:

[0016] capturing a second initial image via the camera assembly; and

[0017] A depth map is calculated based on the plurality of first initial images and the second initial images.

[0018] In some embodiments, in the method, generating the captured image further comprises: performing a bokeh rendering process to introduce a bokeh effect into the captured image according to the depth map.

[0019] In some embodiments, in the method, the plurality of first initial images may be captured by a first primary camera of the camera assembly, and the second initial image may be captured by a second primary camera of the camera assembly.

[0020] In some embodiments, in the method, generating the captured image may further include: performing a segmentation process to introduce a bokeh effect into the captured image based on the intermediate image.

[0021] In some embodiments, in the method, the plurality of first initial images may be captured by a first main camera in a back side of the electronic device or may be captured by a secondary camera in a front side of a camera assembly of the electronic device.

[0022] According to the present application, an electronic device may include:

[0023] A camera assembly capable of capturing images within a first dynamic range; and

[0024] An image processor, the image processor is used to:

[0025] - capturing, by the camera assembly, a plurality of first initial images in a first dynamic range, wherein the plurality of first initial images are captured under different imaging settings;

[0026] - generating an intermediate image in a second dynamic range from the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range;

[0027] - detecting the light source area according to the light detection image to generate a light source map, wherein the light detection image is at least one image among the plurality of first initial images and is suitable for detecting the light source area;

[0028] - generating a captured image based on the intermediate image and the light source map, wherein the position of the light source area of ​​the captured image is determined based on the light source map, and the color of the light source area of ​​the captured image is determined based on the intermediate image.

[0029] In some embodiments, in the electronic device, when capturing a plurality of first initial images, the plurality of first initial images may be captured at different exposure values.

[0030] In some embodiments, in the electronic device, the light detection image may be one of the plurality of first initial images captured at a lowest exposure value.

[0031] In some embodiments, in the electronic device, the minimum exposure value may be low enough not to detect light from an area other than the light source area in the first initial image.

[0032] In some embodiments, in the electronic device, the minimum exposure value may be a negative value.

[0033] In some embodiments, in the electronic device, the image processor may also be used to:

[0034] capturing a second initial image via the camera assembly; and

[0035] A depth map is calculated based on the plurality of first initial images and the second initial images.

[0036] In some embodiments, in the electronic device, when generating a captured image, a bokeh rendering process may also be performed to introduce a bokeh effect into the captured image according to the depth map.

[0037] In some embodiments, in an electronic device, the plurality of first initial images may be captured by a first main camera of a camera assembly, and the second initial image may be captured by a second main camera of the camera assembly.

[0038] In some embodiments, in the electronic device, when generating a captured image, a bokeh effect may also be introduced into the captured image according to the intermediate image through a segmentation process.

[0039] In some embodiments, in the electronic device, the plurality of first initial images may be captured by a first main camera in the back of the electronic device or may be captured by a secondary camera in the front of a camera assembly of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] These and / or other aspects and advantages of the embodiments of the present application will become apparent and more easily understood from the following description with reference to the accompanying drawings, in which:

[0041] Figure 1 A rear view of an electronic device according to a first embodiment of the present application is shown;

[0042] Figure 2 A front view of an electronic device according to a first embodiment of the present application is shown;

[0043] Figure 3 A block diagram of an electronic device according to a first embodiment of the present application is shown;

[0044] Figure 4 A schematic diagram showing a flow chart of an image capturing process performed by an electronic device according to a first embodiment of the present application;

[0045] Figure 5 A rear view of an electronic device according to a second embodiment of the present application is shown; and

[0046] Figure 6 A schematic diagram showing a flow of an image capturing process performed by an electronic device according to a second embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] Embodiments of the present application will be described in detail below, and examples thereof will be shown in the accompanying drawings. Throughout the specification, identical or similar elements and elements having identical or similar functions are represented by identical reference numerals. The embodiments described herein in conjunction with the accompanying drawings are illustrative and intended to explain the present application, and should not be construed as limiting the present application.

[0048] [First embodiment]

[0049] Figure 1 A rear view of an electronic device according to a first embodiment of the present application is shown. Figure 2 A front view of an electronic device according to a first embodiment of the present application is shown.

[0050] like Figure 1 and Figure 2 As shown, the electronic device 10 may include a display 20 and a camera assembly 30. In this embodiment, the camera assembly 30 includes a first main camera 32, a second main camera 34, and a secondary camera 36. The first main camera 32 and the second main camera 34 can capture images on the back of the electronic device 10, while the secondary camera 36 can capture images on the front of the electronic device 10. Therefore, the first main camera 32 and the second main camera 34 are called external cameras, and the secondary camera 36 is called an internal camera.

[0051] The first main camera 32 and the second main camera 34 may have the same performance and / or characteristics, or the first main camera 32 and the second main camera 34 may have different performance and / or characteristics. For example, in the case where the first main camera 32 and the second main camera 34 have different performance and / or characteristics from each other, the first main camera 32 may be equipped with a full-color image sensor, and the second main camera 34 may be equipped with a black and white image sensor. In addition, the first main camera 32 may be a camera suitable for capturing still images, and the second main camera 34 may be a camera suitable for capturing moving images. In addition, the first main camera 32 may be a camera equipped with a wide-angle lens, and the second main camera 34 may be a camera equipped with a telephoto lens.

[0052] In this embodiment, the performance of the secondary camera 36 is lower than that of the first main camera 32 and the second main camera 34. However, the performance of the secondary camera 36 may also be the same as that of the first main camera 32 and the second main camera 34.

[0053] Although the electronic device 10 according to the present embodiment has three cameras, the electronic device may also have more than three cameras. For example, the electronic device 10 may have three, four, five, etc. main cameras.

[0054] Figure 3 FIG. 1 shows a block diagram of an electronic device according to a first embodiment of the present application. Figure 3 As shown, in addition to the display 20 and the camera assembly 30, the electronic device 10 may also include a main processor 40, an image signal processor 42, a memory 44, a power circuit 46, and a communication circuit 48. The display 20, the camera assembly 30, the main processor 40, the image signal processor 42, the memory 44, the power circuit 46, and the communication circuit 48 are connected together via a bus 50.

[0055] The main processor 40 executes one or more programs stored in the memory 44. The main processor 40 implements various applications and data processing of the electronic device 10 by executing the program. The main processor 40 can be one or more computer processors. The main processor 40 is not limited to one CPU core, and can also have multiple CPU cores. The main processor 40 can be the main CPU of the electronic device 10, an image processing unit (IPU), or a DSP provided with a camera assembly 30.

[0056] The image signal processor 42 controls the camera assembly 30 and processes the image captured by the camera assembly 30. For example, the image signal processor 42 can perform de-mosaic processing, noise reduction processing, automatic exposure processing, automatic focus processing, automatic white balance processing, high dynamic range processing, etc. on the image captured by the camera assembly 30.

[0057] In the present embodiment, the main processor 40 and the image signal processor 42 cooperate with each other to obtain the image captured by the camera assembly 30. That is, the main processor 40 and the image signal processor 42 are used to capture the image through the camera assembly 30 and perform various image processing on the captured image.

[0058] The memory 44 stores programs and various data to be executed by the main processor 40. For example, data of a captured image is stored in the memory 44.

[0059] The memory 44 may be a high-speed RAM memory or may be a non-volatile memory such as a flash memory, a disk memory, or the like.

[0060] The power circuit 46 may have a battery (not shown) such as a lithium-ion rechargeable battery and a battery management unit (BMU) for managing the battery.

[0061] The communication circuit 48 is used to receive and send data to communicate with the Internet or other devices via wireless communication. The wireless communication may adopt any communication standard or protocol, including but not limited to: Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), Enhanced Long Term Evolution (LTE-Advanced), and the fifth generation mobile communication system (5G).

[0062] Communications circuitry 48 may include an antenna and radio frequency (RF) circuitry.

[0063] Figure 4 1 is a flow chart showing an image capture process performed by the electronic device according to the present embodiment. In the present embodiment, the image capture process is performed by the main processor 40 in cooperation with the image signal processor 42. Therefore, the main processor 40 and the image signal processor 42 constitute an image processor in the present embodiment.

[0064] like Figure 4 As shown, the electronic device 10 captures multiple first initial images within a first dynamic range through the first main camera 32 of the camera assembly 30 (step S10), and captures a second initial image within the first dynamic range through the second main camera 34 of the camera assembly 30 (step S32).

[0065] In this embodiment, for example, the camera assembly 30 may have an automatic focus setting, an automatic exposure setting, and an automatic white balance setting. In other words, the camera assembly 30 may have an ISO sensitivity setting, an exposure time setting, an exposure value setting, etc.

[0066] The imaging settings of the camera assembly 30 may be automatically set by the electronic device 10. In other words, the user may select automatic settings for the imaging settings of the camera assembly 30. Alternatively, the imaging settings of the camera assembly 30 may be pre-set by the user. In other words, the user may pre-set each item of the imaging settings of the camera assembly 30.

[0067] In addition, in the present embodiment, when the electronic device 10 captures a plurality of first initial images in step S10, the electronic device 10 changes the exposure value of each first initial image to generate an intermediate image within the second dynamic range in a subsequent process. In the present embodiment, the first dynamic range corresponds to a low dynamic range, and the second dynamic range corresponds to a high dynamic range.

[0068] For example, in step S10, the electronic device 10 captures three first initial images, wherein the exposure values ​​are respectively set to -1, 0, and +1. That is, it is assumed that the electronic device 10 has set the exposure value of the camera assembly 30 to 0. In order to capture another two first initial images, the exposure value is increased by 1 and decreased by 1, respectively.

[0069] However, the number of the first initial images is optional. For example, the number of the first initial images captured in step S10 may be two, four, five, etc.

[0070] When capturing the second initial image in step S12, the electronic device 10 captures an image of the second initial image to calculate the depth map in subsequent processing. In this embodiment, although the plurality of first initial images and the plurality of second initial images are captured within the first dynamic range, the dynamic range of the first initial image may be different from the dynamic range of the second initial image. In other words, the first dynamic range is the dynamic range of the first main camera 32 and the second main camera 34.

[0071] In this embodiment, the first initial image with an exposure value of 0 and the second initial image are captured simultaneously because the exposure value at 0 is an imaging setting set by the electronic device 10. However, the capture time of the first initial image with an exposure value of 0 and the capture time of the second initial image may be different.

[0072] Next, the electronic device 10 performs a high dynamic range generation process (step S14 ). More specifically, in the high dynamic range generation process, the electronic device 10 generates an intermediate image in the second dynamic range from a plurality of first initial images.

[0073] In this embodiment, the electronic device 10 generates an intermediate image based on three first initial images respectively captured at exposure values ​​of -1, 0 and 1. An intermediate image with a high dynamic range may be generated by combining and synthesizing a plurality of first initial images with a low dynamic range.

[0074] Next, if Figure 4 As shown, the electronic device 10 calculates a depth map based on the intermediate image generated in step S14 and the second initial image captured in step S12 (step S16).

[0075] More specifically, the position of the first main camera 32 is different from the position of the second main camera 34. Therefore, the viewpoint of the intermediate image generated based on the first initial image captured by the first main camera 32 is different from the viewpoint of the second initial image captured by the second main camera 34. Using the parallax between the intermediate image and the second initial image, the electronic device 10 can generate a depth map indicating the distance between the electronic device 10 and the surface of the object in the intermediate image and the second main image.

[0076] For example, the depth map may be calculated by the main processor 40 and / or the image signal processor 42. In addition, the depth map may be calculated inside the camera module of the camera assembly 30. However, a depth map calculation circuit for calculating the depth map may be placed in the electronic device 10.

[0077] Next, if Figure 4 As shown, the electronic device 10 detects the light source area according to the light detection image, and generates a light source map by detecting the light source in the light detection image to indicate the position of the light source area in the intermediate image (step S18).

[0078] The light source map may be an image indicating the centers of light sources or may be a set of one or more coordinates of the centers of light sources on the captured image. The light source map may be temporarily stored in the memory 44 .

[0079] The light detection image is at least one of the plurality of first initial images and is suitable for detecting the light source area. For example, in this case, one of the first initial images captured with an exposure value of -1 is the light detection image because the image captured with an exposure value of -1 is the image with the lowest exposure value among the plurality of first initial images captured in step S10. In other words, the exposure value of -1 is low enough so that light from an area other than the light source area cannot be detected in the first initial image.

[0080] Generally, in order to detect the light source area in the light detection image, the exposure value (EV) is preferably a negative value. That is, the light detection image should be one of the first initial images captured at a negative exposure value. Here, the exposure value can be composed of shutter speed, ISO sensitivity and aperture.

[0081] In this embodiment, in order to generate a light source map, the electronic device 10 compares the brightness of a certain area in the light detection image with a threshold value. If the brightness of the area is greater than the threshold value, the area is regarded as a light source area.

[0082] More specifically, the electronic device 10 compares the brightness of each area in the light detection image with the threshold. In other words, the electronic device 10 compares the brightness of each pixel in the light detection image with the threshold to detect the light source area in the light detection image. The threshold can be stored in the memory 44, set in a program executed by the main processor 40, or set in the image signal processor 42.

[0083] For example, in the case where each pixel in the light detection image includes brightness consisting of 256 gray levels, if the brightness value of the pixel is greater than 128, the electronic device 10 can determine that the pixel is in the light source area.

[0084] Incidentally, the threshold value is not necessarily set to a specific value, but may also be set to a percentage. For example, in a case where the maximum brightness is 100% and the minimum brightness is 0%, the threshold value may be set to 50% of the brightness.

[0085] Thereafter, the electronic device 10 detects the center of the area where the brightness is greater than the threshold value and determines the center as the center of the light source to generate a light source map.

[0086] In addition, in this embodiment, although the electronic device 10 detects the light source area in the light detection image after calculating the depth map, the electronic device 10 may detect the light source area in the light detection image before calculating the depth map. In this case, step S18 is performed before step S16.

[0087] For example, the light source area may be detected based on the light detection image by the main processor 40 and / or the image signal processor 42. In addition, the light source area may be detected based on the light detection image captured by the camera assembly 30.

[0088] Next, if Figure 4 As shown, the electronic device 10 performs image generation processing to generate a captured image (step S20). In this embodiment, the image generation processing is performed at least based on the intermediate image generated in step S14, the light source map generated in step S18, and the depth map calculated in step S16. Of course, in the image generation processing of step S20, other images and / or any information can be used to generate the captured image.

[0089] In this embodiment, the image generation process includes at least a light source region adjustment process, which appropriately renders the light source region onto the captured image. In the light source region adjustment process, the position of the light source region is determined based on the light source map, and the color of the light source region is determined based on the intermediate image.

[0090] In addition, the image generation process includes at least a bokeh rendering process based on a depth map. That is, after an intermediate image is generated based on the plurality of first initial images, a bokeh rendering process is applied to the intermediate image to defocus the background of the intermediate image based on the depth map. In other words, the background of the intermediate image is defocused to become blurred by the bokeh rendering process.

[0091] Generally speaking, the color of the intermediate image with a high dynamic range is more natural and accurate than the color of each of the first initial images and each of the second initial images. On the other hand, if the bokeh rendering process is applied to the intermediate image with a high dynamic range, the bokeh effect will also be introduced into the light source area, making the light source area too blurred. Therefore, in this embodiment, the position of the light source area is adjusted according to the light source map generated in step S18, and the color of the light source area is adjusted according to the color of the intermediate image.

[0092] When the image generation process in step S20 is completed, a captured image that the user wants to capture is generated. Of course, the image generation process for generating the captured image may include various processes applied to the intermediate image in addition to the bokeh rendering process and the light source area adjustment process. For example, the image generation process may include a demosaic process, a noise reduction process, an automatic exposure process, an automatic focus process, an automatic white balance process, a high dynamic range process, etc., to generate the captured image.

[0093] For example, the image generation process may be performed by the main processor 40 and / or the image signal processor 42. However, a bokeh rendering circuit for performing the bokeh rendering process may be placed in the electronic device 10. Similarly, a light source area adjustment circuit for performing the light source area adjustment process may be placed in the electronic device 10.

[0094] Next, if Figure 4 As shown, the electronic device 10 outputs the captured image (step S22). For example, the electronic device 10 may display the captured image on the display 20. In addition, the electronic device 10 may store the captured image in the memory 44.

[0095] After the electronic device 10 outputs the captured image in step S22 , the image capturing process of this embodiment is completed.

[0096] As described above, according to the electronic device 10 of this embodiment, when the electronic device 10 generates a captured image, the position of the light source area on the captured image is determined according to the light source diagram and the color of the captured image, and the light source area on the captured image is determined according to the intermediate image of the high dynamic range. Therefore, the light source area on the captured image is more beautiful and more natural. In this way, the user can be satisfied with the quality of the captured image of the electronic device 10 according to this embodiment.

[0097] In addition, since one of the plurality of first initial images can be used as a light detection image to detect the light source area, the camera assembly 30 is not required to capture additional images or additional mechanical elements are not required. Therefore, user satisfaction with captured images can be improved without any additional cost.

[0098] Although the above-described electronic device 10 changes the exposure value for capturing each of the plurality of first initial images in step S10 so as to generate an intermediate image within a high dynamic range in step S14, the electronic device 10 may change imaging settings other than the exposure value for capturing each of the plurality of first initial images. For example, the electronic device 10 may change the ISO sensitivity, the exposure time, etc., so as to capture each of the plurality of first initial images in step S10.

[0099] In any case, the most suitable image for detecting the light source area and generating the light source map in the first initial image can be used as the light detection image in step S18. In other words, since the electronic device 10 captures multiple first initial images in order to generate an intermediate image with a high dynamic range, the image that is most suitable for detecting the light source area can be selected as the light detection image in step S18.

[0100] [Second embodiment]

[0101] Although the electronic device 10 has two main cameras on its back in the first embodiment of the present application, in the second embodiment of the present application, the electronic device 10 has one main camera on its back. Hereinafter, the differences from the first embodiment will be described.

[0102] Figure 5 FIG. 1 shows a rear view of an electronic device 10 according to a second embodiment of the present application. Figure 5 Compared with the first embodiment Figure 1 The front view of the electronic device 10 according to the second embodiment is similar to that of the first embodiment. Figure 2 About the same.

[0103] like Figure 5 As shown, the electronic device 10 according to this embodiment has a camera assembly 30, which includes a main camera 38 but does not include other main cameras. That is, the camera assembly 30 of the electronic device 10 according to this embodiment has a main camera 38 on the back of the electronic device 10 to capture images on the back of the electronic device 10.

[0104] like Figure 2As shown, in the second embodiment, similar to the first embodiment, the electronic device 10 has a secondary camera 36. That is, the camera assembly 30 of the electronic device 10 according to the second embodiment further includes a built-in camera on the front of the electronic device 10 so that it can capture images on the front.

[0105] Figure 6 1 is a flow chart of an image capture process performed by an electronic device according to a second embodiment of the present application. In this embodiment, the image capture process is performed by a main processor 40 in cooperation with an image signal processor 42. Therefore, the main processor 40 and the image signal processor 42 constitute an image processor in this embodiment.

[0106] like Figure 6 As shown, the electronic device 10 captures a plurality of first initial images within a first dynamic range through the main camera 38 of the camera assembly 30 (step S30), wherein the first dynamic range is lower than the second dynamic range. Except that the plurality of first initial images are captured by a single main camera 38, step S30 in the second embodiment is substantially the same as step S10 in the first embodiment.

[0107] On the other hand, since the camera assembly 30 of the electronic device 10 of the second embodiment does not have the second main camera 34, the second initial image is not captured. In other words, the step S12 of capturing the second initial image in the first embodiment is omitted in the second embodiment.

[0108] Next, if Figure 6 As shown, the electronic device 10 performs a high dynamic range generation process to generate an intermediate image with a high dynamic range based on a plurality of first initial images with a low dynamic range (step S14). Step S14 in the second embodiment is substantially the same as the corresponding step in the first embodiment.

[0109] Next, if Figure 6 As shown, the electronic device 10 detects the light source area according to the light detection image to generate a light source map by detecting the light source in the light detection image, thereby indicating the position of the light source area in the intermediate image (step S18). Step S18 in the second embodiment is substantially the same as the corresponding step in the first embodiment.

[0110] Next, if Figure 6 As shown, the electronic device 10 performs image generation processing to generate a captured image (step S32). In this embodiment, the image generation processing is performed at least based on the intermediate image generated in step S14 and the light source diagram generated in step S18. Of course, in the image generation processing of step S32, other images and / or any information can be used to generate the captured image.

[0111] However, the depth map is not used in the image generation process according to the present embodiment because the camera assembly 30 includes a single main camera 38. That is, step S16 of generating the depth map in the first embodiment is omitted in the second embodiment.

[0112] Therefore, the image generation process according to the present embodiment includes a segmentation process instead of a bokeh rendering process. In the segmentation process, a bokeh effect is introduced into the captured image without the depth map calculated in step S16 of the first embodiment. For example, the bokeh effect is introduced into the captured image by analyzing and recognizing a human face in the intermediate image, segmenting a human silhouette, and introducing the bokeh effect into the background of the human silhouette in the intermediate image.

[0113] Similar to the first embodiment, the image generation process according to this embodiment includes a light source region adjustment process, that is, in this embodiment, the position of the light source region is determined based on the light source map, and the color of the light source region is determined based on the intermediate image.

[0114] Next, if Figure 4 As shown, the electronic device 10 outputs the captured image (step S22). Step S22 in the second embodiment is substantially the same as the corresponding step in the first embodiment.

[0115] As described above, according to the electronic device 10 of this embodiment, when the electronic device 10 generates a captured image, the position of the light source area on the captured image is determined according to the light source map, and the light source color on the captured image is determined according to the intermediate image of the high dynamic range. Therefore, the light source area on the captured image can be more beautiful and natural. In this way, the user can be satisfied with the quality of the captured image captured by the electronic device 10 according to this embodiment.

[0116] In addition, even if the electronic device 10 includes a single main camera 38, the electronic device 10 can perform image generation processing including light source area adjustment processing. Therefore, the present embodiment can be implemented by the camera assembly 30 and the single main camera 38 of the electronic device 10.

[0117] Similarly, this embodiment can be implemented by the secondary camera 36 on the front of the electronic device 10. In other words, if the secondary camera 36 captures multiple first initial images, the operation can be performed according to the multiple first initial images captured by the secondary camera 36. Figure 6 Therefore, the quality of the captured image captured by the secondary camera 36 can also be improved.

[0118] In the description of the embodiments of the present application, it should be understood that terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise" and "counterclockwise" should be understood to refer to the direction or position shown in the described or discussed drawings. These relative terms are only used to simplify the description of the present application and do not indicate or imply that the mentioned device or element must have a specific direction, or be constructed or operated in a specific direction. Therefore, these terms should not be interpreted as limiting the present application.

[0119] In addition, terms such as "first" and "second" are used herein for descriptive purposes and are not intended to indicate or imply relative importance or significance, or to imply the number of technical features indicated. Therefore, features defined with "first" and "second" may include one or more of such features. In the description of this application, "plurality" means two or more, unless otherwise specified.

[0120] In the description of the embodiments of the present application, unless otherwise specified or limited, terms such as "installation", "connection", and "coupling" are broad in nature, and may mean, for example, a fixed connection, a detachable connection, or an integral connection; it may also be a mechanical or electrical connection; it may be a direct connection or an indirect connection through an intermediate structure; it may also be internal communication between two elements, and those skilled in the art may understand this based on the specific circumstances.

[0121] In the embodiments of the present application, unless otherwise specified or limited, the structure in which the first feature is "on" or "below" the second feature may include an embodiment in which the first feature is in direct contact with the second feature, and may also include an embodiment in which the first feature and the second feature are not in direct contact with each other, but are in contact through an additional feature formed between them. In addition, the first feature being "on", "above" or "top" of the second feature may include the first feature being straight or obliquely "on", "above" or "top" of the second feature, or simply indicating that the height of the first feature is higher than the height of the second feature; and the first feature being "below", "below" or "bottom" of the second feature may include the first feature being exactly or obliquely "below", "below" or "bottom" of the second feature, or simply indicating that the height of the first feature is lower than the height of the second feature.

[0122] Various embodiments and examples are provided in the above description to realize different structures of the present application. In order to simplify the present application, certain elements and settings are described above. However, these elements and settings are only used as examples and are not intended to limit the present application. In addition, reference numerals and / or reference letters may be repeated in different examples of the present application. This repetition is for the purpose of simplification and clarity, rather than referring to the relationship between different embodiments and / or settings. In addition, examples of different processes and materials are provided in the present application. However, those skilled in the art will appreciate that other processes and / or materials may also be applied.

[0123] References throughout this specification to "one embodiment," "some embodiments," "an exemplary embodiment," "an example," "a specific example," or "some examples" mean that a particular feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. Therefore, the appearance of the above phrases in this specification does not necessarily refer to the same embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0124] Any process or method described in a flowchart or otherwise described herein may be understood to include one or more modules, fragments or code portions of executable instructions for implementing specific logical functions or steps in the process, and the scope of the preferred embodiments of the present application includes other implementations. Those skilled in the art should understand that the functions in these implementations may be implemented in an order different from that shown or discussed, including in substantially the same order or in reverse order.

[0125] The logic and / or steps described in other ways herein or shown in the flowcharts, such as a specific sequence list of executable instructions for implementing the logic functions, can be specifically implemented in any computer-readable medium, which is used by an instruction execution system, device or apparatus (for example, a computer-based system, a system including a processor, or other system capable of obtaining instructions from an instruction execution system, device and apparatus and executing the instructions), or a combination of instruction execution systems, devices and apparatuses. For the purpose of this specification, a "computer-readable medium" can be any device suitable for containing, storing, communicating, propagating or transmitting a program for use by an instruction execution system, device or apparatus, or a combination of instruction execution systems, devices and apparatuses. More specific examples of computer-readable media include, but are not limited to: an electronic connection with one or more wires (electronic devices), a portable computer housing (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because, for example, the program can be obtained electronically if necessary, the paper or other suitable medium can be optically scanned, and then edited, decrypted or processed by other suitable methods, and then the program can be stored in the computer memory.

[0126] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by an appropriate instruction execution system. For example, if it is implemented by hardware, in another embodiment, these steps or methods can be implemented by one or a combination of the following technologies known in the art: a discrete logic circuit with a logic gate circuit for implementing the logic function of a data signal, a dedicated integrated circuit with an appropriate combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0127] Those skilled in the art should understand that all or part of the steps in the above exemplary method of the present application can be implemented by commanding relevant hardware through a program. The program can be stored in a computer-readable storage medium, and when the program is run on a computer, it includes a combination of one or more steps in the method embodiment of the present application.

[0128] In addition, each functional unit in the embodiment of the present application may be integrated into a processing module, or may be physically independent of each other, or two or more units may be integrated into a processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, the integrated module may be stored in a computer-readable storage medium.

[0129] The above storage medium may be a read-only memory, a disk, a CD, etc.

[0130] Although embodiments of the present application have been shown and described, those skilled in the art should understand that these embodiments are illustrative and cannot be construed as limiting the present application, and that changes, modifications, substitutions and variations may be made to the present application without departing from the scope of the present application.

Claims

1. A method for generating a captured image in an electronic device, the electronic device comprising a camera assembly, the method comprising: capturing, by the camera assembly, a plurality of first initial images within a first dynamic range, wherein the plurality of first initial images are captured under different imaging settings, and the first initial images are captured by a first main camera of the camera assembly; generating an intermediate image of a second dynamic range according to the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range; Detecting a light source area according to a light detection image to generate a light source map, wherein the light detection image is at least one image among the plurality of first initial images and is suitable for detecting the light source area; generating the captured image according to the intermediate image and the light source map, wherein the position of the light source area of ​​the captured image is determined according to the light source map, and the color of the light source area of ​​the captured image is determined according to the intermediate image; capturing a second initial image by the camera assembly, the second initial image being captured by a second main camera of the camera assembly, the dynamic range of the first initial image being different from the dynamic range of the second initial image; Calculating a depth map based on the intermediate image and the second initial image; The generating the captured image further comprises: A bokeh rendering process is performed to introduce a bokeh effect into the captured image based on the depth map.

2. The method according to claim 1, characterized in that The capturing of the plurality of first initial images comprises: capturing the plurality of first initial images at different exposure values.

3. The method according to claim 2, characterized in that The light detection image is an image captured at a lowest exposure value among the plurality of first initial images.

4. The method according to claim 3, characterized in that The minimum exposure value is low enough to detect light from an area other than the light source area in the first initial image.

5. The method according to claim 4, characterized in that The minimum exposure value is a negative value.

6. The method according to any one of claims 1 to 5, characterized in that Generating the captured image further includes performing a segmentation process to introduce a bokeh effect into the captured image based on the intermediate image.

7. An electronic device comprising: a camera assembly capable of capturing images within a first dynamic range; and An image processor, the image processor being configured to: capturing, by the camera assembly, a plurality of first initial images within the first dynamic range, wherein the plurality of first initial images are captured under different imaging settings, and the plurality of first initial images are captured by a first main camera of the camera assembly; generating an intermediate image of a second dynamic range according to the plurality of first initial images, wherein the second dynamic range is higher than the first dynamic range; Detecting a light source area according to a light detection image to generate a light source map, wherein the light detection image is at least one image among the plurality of first initial images and is suitable for detecting the light source area; generating a captured image according to the intermediate image and the light source map, wherein a position of the light source area of ​​the captured image is determined according to the light source map, and a color of the light source area of ​​the captured image is determined according to the intermediate image; capturing a second initial image by the camera assembly, the second initial image being captured by a second main camera of the camera assembly, the dynamic range of the first initial image being different from the dynamic range of the second initial image; Calculating a depth map based on the intermediate image and the second initial image; When generating the captured image, a bokeh rendering process is also performed to introduce a bokeh effect into the captured image according to the depth map.

8. The electronic device according to claim 7, characterized in that: When capturing the plurality of first initial images, the plurality of first initial images are captured at different exposure values.

9. The electronic device according to claim 8, characterized in that: The light detection image is an image captured at a lowest exposure value among the plurality of first initial images.

10. The electronic device according to claim 9, characterized in that: The minimum exposure value is low enough to detect light from an area other than the light source area in the first initial image.

11. The electronic device according to claim 10, characterized in that: The minimum exposure value is a negative value.

12. The electronic device according to any one of claims 7 to 11, characterized in that: When generating the captured image, a bokeh effect is also introduced into the captured image based on the intermediate image through a segmentation process.

Images