Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, electronic equipment and storage medium. The method comprises: in response to detecting a shooting request, obtaining exposure information of a current shooting scene, the shooting request comprising the exposure information, the exposure information comprising a plurality of exposure values, the exposure values being used to represent the exposure degree of an exposure image; based on the exposure information, obtaining an exposure image corresponding to each exposure value respectively; and based on a plurality of exposure images, performing fusion to obtain an output image. This output image is obtained by shooting exposure images with different exposure degrees according to one shooting request, thereby reducing the time difference between adjacent two exposure images and the displacement between adjacent two exposure images, so as to avoid ghosting or ghosting problems when fusing a plurality of exposure images, and improve the quality of the output image obtained by fusion.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and in particular, to an image processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the wide use of various electronic devices, the use rate of the photographing function of the electronic devices gradually increases, and the requirement for image quality is also increasingly high. As a common photographing function, the high dynamic range (HDR) technology can provide a high-contrast effect image for users, restore a dynamic range scene, improve image quality, and bring a very good image quality experience to users. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides an image processing method and device, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, and the method comprises:

[0005] In response to detecting a photographing request, exposure information of a current photographing scene is acquired, the photographing request comprising the exposure information, the exposure information comprising a plurality of exposure values, the exposure values being used to represent exposure degrees of exposure images;

[0006] Based on the exposure information, an exposure image corresponding to each of the exposure values is acquired respectively;

[0007] Based on a plurality of the exposure images, a fusion is performed to obtain an output image.

[0008] In some embodiments, the exposure image corresponding to each of the exposure values is acquired respectively based on the exposure information, comprising:

[0009] Based on the exposure information and a preset relationship, an exposure time and a gain corresponding to each of the exposure values are determined respectively, the preset relationship being used to represent a relationship between an exposure value and an exposure time and a gain;

[0010] Based on the exposure time and the gain corresponding to each of the exposure values, the exposure image corresponding to each of the exposure values is acquired respectively.

[0011] In some embodiments, before the exposure image corresponding to each of the exposure values is acquired respectively based on the exposure time and the gain corresponding to each of the exposure values, the method further comprises:

[0012] The gain corresponding to each of the exposure values is adjusted respectively to obtain an adjusted gain corresponding to each of the exposure values, wherein the plurality of adjusted gains are the same;

[0013] adjust, based on an adjustment range of the gain corresponding to each of the exposure values, an exposure time corresponding to each of the exposure values, to obtain an adjusted exposure time corresponding to each of the exposure values.

[0014] In some embodiments, the acquiring, based on the exposure information, an exposure image corresponding to each of the exposure values respectively comprises:

[0015] acquiring image data of each row based on the exposure information, and saving the image data when the image data of each row is acquired, until the image data of the last row is saved;

[0016] determining, based on the saved image data, an exposure image corresponding to each of the exposure values.

[0017] In some embodiments, the image data of each row comprises image sub-data corresponding to different exposure values; and the saving the image data comprises:

[0018] saving each of the image sub-data to a corresponding data storage area, the number of the data storage areas being the same as the number of the exposure values;

[0019] The determining, based on the saved image data, an exposure image corresponding to each of the exposure values comprises:

[0020] reading the saved image sub-data from each of the data storage areas to determine an exposure image corresponding to each of the exposure values.

[0021] In some embodiments, the image format of the exposure image is a first format; and after the acquiring, based on the exposure information, an exposure image corresponding to each of the exposure values respectively, the method further comprises:

[0022] performing format conversion processing on each of the exposure images to obtain an exposure image in a second format, the second format being different from the first format.

[0023] In some embodiments, the fusing, based on a plurality of the exposure images, to obtain an output image comprises:

[0024] fusing, based on a plurality of the exposure images and a weight corresponding to each of the exposure images, to obtain the output image.

[0025] In some embodiments, before the acquiring, in response to detecting a shooting request, exposure information of a current shooting scene, the method further comprises:

[0026] In response to detecting a shooting instruction, generating the shooting request.

[0027] According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is provided, the apparatus comprising:

[0028] an exposure information acquisition module configured to acquire exposure information of a current shooting scene in response to detecting a shooting request, the shooting request comprising the exposure information, the exposure information comprising a plurality of exposure values, the exposure values being used to represent exposure degrees of exposure images;

[0029] an exposure image acquisition module configured to acquire an exposure image corresponding to each of the exposure values based on the exposure information;

[0030] an output image acquisition module configured to obtain an output image by fusing a plurality of the exposure images.

[0031] In some embodiments, the exposure image acquisition module is configured to:

[0032] determine an exposure time and a gain corresponding to each of the exposure values based on the exposure information and a preset relationship, the preset relationship being used to represent a relationship between an exposure value and an exposure time and a gain;

[0033] acquire an exposure image corresponding to each of the exposure values based on the exposure time and the gain corresponding to each of the exposure values.

[0034] In some embodiments, the exposure image acquisition module is further configured to:

[0035] adjust the gain corresponding to each of the exposure values to obtain an adjusted gain corresponding to each of the exposure values, wherein the adjusted gains are the same;

[0036] adjust the exposure time corresponding to each of the exposure values based on an adjustment amplitude of the gain corresponding to each of the exposure values to obtain an adjusted exposure time corresponding to each of the exposure values.

[0037] In some embodiments, the exposure image acquisition module is configured to:

[0038] acquire image data of each row in sequence based on the exposure information, save the image data when the image data of each row is acquired, and save the image data of the last row until the image data of the last row is saved;

[0039] determine an exposure image corresponding to each of the exposure values based on the saved image data.

[0040] In some embodiments, the image data of each row comprises image sub-data corresponding to different exposure values, and the exposure image acquisition module is configured to:

[0041] save each of the image sub-data to a corresponding data storage area respectively, a number of the data storage areas being same as a number of the exposure values;

[0042] read the saved image sub-data from each of the data storage areas, and determine an exposure image corresponding to each of the exposure values.

[0043] In some embodiments, an image format of the exposure image is a first format; and the apparatus further includes:

[0044] a format conversion module configured to, for each of the exposure images, perform a format conversion process on the exposure image to obtain an exposure image in a second format, the second format being different from the first format.

[0045] In some embodiments, the output image obtaining module is configured to:

[0046] fuse, based on a plurality of the exposure images and a weight corresponding to each of the exposure images, to obtain the output image.

[0047] In some embodiments, the apparatus further includes:

[0048] a request generating module configured to, in response to detecting a photographing instruction, generate the photographing request.

[0049] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, including:

[0050] a processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to perform the method as described in the first aspect of embodiments of the present disclosure.

[0053] According to a fourth aspect of embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as described in the first aspect of embodiments of the present disclosure.

[0054] By using the above method of the present disclosure, the following beneficial effects are achieved:

[0055] The method provided by the embodiment of the present disclosure, in response to detecting a shooting request, acquires exposure information of a current shooting scene, the shooting request comprising the exposure information, the exposure information comprising a plurality of exposure values, the exposure values being used to represent exposure degrees of exposure images; based on the exposure information, exposure images corresponding to each exposure value are acquired respectively; and based on the plurality of exposure images, an output image is obtained by fusion. This output image generation manner can obtain a plurality of exposure images of different exposure degrees at one time according to one shooting request, reduce the time difference between adjacent two exposure images, and reduce the displacement between adjacent two exposure images, thereby avoiding ghosting or ghosting problems when the plurality of exposure images are fused, and improving the quality of the output image obtained by fusion.

[0056] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0057] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0058] Figure 1 is a schematic diagram of an output image generation manner in a related technology according to an exemplary embodiment;

[0059] Figure 2 is a schematic diagram of another output image generation manner in a related technology according to an exemplary embodiment;

[0060] Figure 3 is a flowchart of an image processing method according to an exemplary embodiment;

[0061] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment;

[0062] Figure 5 is a flowchart of another image processing method according to an exemplary embodiment;

[0063] Figure 6 is a schematic diagram of an output image generation manner according to an exemplary embodiment;

[0064] Figure 7 is a flowchart of ISP processing according to an exemplary embodiment;

[0065] Figure 8 is a schematic diagram of exposure image fusion according to an exemplary embodiment;

[0066] Figure 9is a block diagram of an image processing apparatus according to an exemplary embodiment;

[0067] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0068] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar elements, unless otherwise represented. The embodiments described in the following exemplary embodiments do not represent all the implementations in accordance with this disclosure. Instead, they simply represent exemplary devices and methods in accordance with some aspects of this disclosure, as detailed in the appended claims.

[0069] HDR is generally to fuse a high dynamic range image using multiple images of different exposure levels. It is also a great challenge for HDR technology to have good image quality experience in different scenes. For example, when there is a moving object in the scene, the user wants to capture the current moving object, and the moving object may be displaced too much or not in the image, resulting in poor user experience.

[0070] In one related technology, a frame-out image output method is adopted to output exposure images: receiving multiple shooting requests, and outputting different exposure images according to each shooting request. For example, see Figure 1 , Figure 1 In FIG. 1, the horizontal coordinate represents time, and the vertical coordinate represents the row number, which refers to the row number of the pixel points in the image. When the image is output, the exposure images are output frame by frame, and N, N+1, N+2, etc. represent 1 frame of exposure image. In this image output method, the time difference between two adjacent exposure images is the difference between the exposure start time points of the two exposure images.

[0071] In another related technology, a line-out image output method is adopted to output exposure images: receiving one shooting request, and outputting different exposure images according to the shooting request. The shooting request carries multiple request information, and each request information is used to request output of one frame of exposure image. For example, see Figure 2 , Figure 2 In FIG. 2, the horizontal coordinate represents time, and the vertical coordinate represents the row number. When the image is output, multiple exposure images are output simultaneously, Figure 2 In FIG. 2, the horizontal coordinate represents time, and the vertical coordinate represents the row number. When the image is output, multiple exposure images are output simultaneously,

[0072] In the above two related technologies, when multiple exposure images are acquired, there is a long time difference between adjacent two exposure images, that is, the displacement between adjacent two exposure images is large, thereby causing ghosting or ghosting problem when the multiple exposure images are fused, resulting in poor quality of the output image fused.

[0073] The embodiment of the present disclosure proposes a new image processing method, which can reduce the time difference between adjacent two exposure images, that is, reduce the displacement between adjacent two exposure images, thereby avoiding the ghosting or ghosting problem when the multiple exposure images are fused, and improving the quality of the output image fused.

[0074] The method provided by the embodiment of the present disclosure is executed by an electronic device, which can be a mobile phone, a tablet computer, a notebook computer, a camera device, or the like.

[0075] Figure 3 is a flowchart of an image processing method according to an exemplary embodiment, executed by an electronic device, see Figure 3 The method comprises the following steps:

[0076] In step S301, in response to detecting a shooting request, exposure information of a current shooting scene is acquired, the shooting request comprising the exposure information, the exposure information comprising a plurality of exposure values, the exposure value being used to represent the exposure degree of an exposure image.

[0077] The shooting request is used to request to shoot multiple exposure images, and the exposure information comprises a plurality of exposure values (Exposure Value, EV), which is used to represent the exposure degree of an exposure image. For example, the exposure information comprises three exposure values EV0, EV- and EV+, EV0 represents normal exposure (normal exposure), EV- represents low exposure (dark), and EV+ represents high exposure (bright).

[0078] Since the shooting request comprises the exposure information, in response to detecting the shooting request, the exposure information of the current shooting scene can be acquired from the shooting request. The current shooting scene is the current shot picture.

[0079] In step S302, based on the exposure information, an exposure image corresponding to each exposure value is acquired.

[0080] Since the exposure information includes multiple exposure values, a corresponding exposure image can be acquired according to each exposure value. The exposure image corresponding to each exposure value is acquired according to the exposure degree represented by the exposure value, and the exposure images corresponding to different exposure values exhibit different exposure degrees. Moreover, when the exposure images are acquired, multiple exposure values can be used for acquisition at the same time, so as to reduce the time difference between different exposure images during acquisition and reduce the displacement between adjacent two exposure images.

[0081] In step S303, the multiple exposure images are fused to obtain an output image.

[0082] The fusion based on the multiple exposure images, that is, the exposure images under different exposure degrees are comprehensively considered to obtain the output image, the image information under different exposure degrees is comprehensively considered in the output image, the image quality of the output image is better, and more image information is included compared with a single exposure image.

[0083] The method provided by the embodiment of the present disclosure includes the following steps: in response to detecting a shooting request, exposure information of a current shooting scene is acquired, the shooting request includes the exposure information, the exposure information includes multiple exposure values, and the exposure values are used to represent exposure degrees of exposure images; based on the exposure information, exposure images corresponding to each exposure value are acquired respectively; and based on the multiple exposure images, fusion is performed to obtain an output image. This image output method can obtain multiple exposure images at one time according to one shooting request, which requests to shoot exposure images with different exposure degrees, reduces the time difference between adjacent two exposure images, reduces the displacement between adjacent two exposure images, avoids ghosting or ghosting problems during fusion of the multiple exposure images, and improves the quality of the output image obtained by fusion.

[0084] The image processing method provided by the embodiment of the present disclosure includes multiple frame requests, multiple frame image acquisition, ISP (Image Signal Processor, image signal processor) processing, multiple frame algorithm fusion, and output Figure five The following embodiments are used to describe the above-mentioned stages respectively.

[0085] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment, executed by an electronic device, see Figure 4 The method includes the following steps:

[0086] In step S401, in response to detecting a shooting request, exposure information of a current shooting scene is acquired.

[0087] The shooting request includes the exposure information, the exposure information includes multiple exposure values, and the exposure values are used to represent exposure degrees of exposure images.

[0088] In some embodiments, in response to detecting the shooting instruction, the shooting request is generated. For example, the user clicks the shooting button, the electronic device detects the click operation of the user on the shooting button, determines that the shooting instruction is detected, and generates the shooting request.

[0089] Optionally, the exposure value can be randomly set or calculated by using a preset algorithm. For example, the electronic device is configured with a preset algorithm, and in the process of previewing the image, the exposure information is calculated by using the preset algorithm based on the current environmental brightness, color temperature, photosensitivity, gain (Gain), and current shooting scene information. Of course, other ways of determining the exposure value can also be used, and the embodiments of the present disclosure do not limit this.

[0090] In step S402, the exposure time and the gain corresponding to each exposure value are respectively determined based on the exposure information and a preset relationship.

[0091] In actual collection of the exposure image, the collection needs to be performed according to the exposure time and the gain. The exposure time refers to the time interval from the opening to the closing of the shutter. In the case of no overexposure, increasing the exposure time can increase the signal-to-noise ratio and make the image clearer. The gain refers to the amplification gain of the analog signal after double sampling.

[0092] The preset relationship is used to represent the relationship between the exposure value and the exposure time and the gain. Optionally, the preset relationship is a first preset function, and the exposure time and the gain are determined according to the exposure value and the first preset function. Alternatively, the preset relationship is a preset first corresponding relationship, and the first corresponding relationship includes the exposure value and the corresponding exposure time and gain.

[0093] In some embodiments, an Auto Exposure (AE) module is included in the electronic device, and the AE module is used to convert the exposure value to obtain the exposure time and the gain corresponding to the exposure value. For example, the exposure time and the gain after conversion of the exposure information (EV0, EV-10, EV+8) are as follows: ExposureTime: Gain (10000000: 1.2, 8000000: 1.0, 15000000: 2.0). The unit of the exposure time is nanosecond.

[0094] Optionally, in order to facilitate subsequent acquisition of exposure images, the gain corresponding to different exposure values can be made the same, that is, the gain corresponding to each exposure value is adjusted respectively to obtain an adjusted gain corresponding to each exposure value, wherein the plurality of adjusted gains are the same; based on the adjustment range of the gain corresponding to each exposure value, the exposure time corresponding to each exposure value is adjusted to obtain an adjusted exposure time corresponding to each exposure value. That is, the gain corresponding to each exposure value is adjusted to the same value, and the exposure time corresponding to each exposure value is adjusted according to the adjustment range of the corresponding gain, so that the exposure degree represented by the adjusted set of exposure times and gains is the same as before adjustment.

[0095] Optionally, based on the adjustment range of the gain corresponding to each exposure value, adjusting the exposure time corresponding to each exposure value comprises: for each exposure value, adjusting the exposure time corresponding to the exposure value according to the adjustment range of the corresponding gain to obtain an adjusted exposure time.

[0096] For example, the exposure time and gain: ExposureTime: Gain (10000000: 1.2, 8000000: 1.0, 15000000: 2.0) is adjusted to: ExposureTime: Gain (12000000: 1.0, 8000000: 1.0, 30000000: 1.0), and the adjusted gain is 1.

[0097] Step S403, based on the exposure time and gain corresponding to each exposure value, an exposure image corresponding to each exposure value is obtained respectively.

[0098] The exposure time and gain corresponding to each exposure value can be before adjustment or after adjustment.

[0099] Step S404, based on the plurality of exposure images, an output image is obtained by fusion.

[0100] The method provided by the embodiments of the present disclosure converts exposure values into exposure times and gains, and acquires exposure images based on the exposure times and gains. This exposure method can obtain a plurality of exposure images at one time according to one shooting request, which reduces the time difference between adjacent two exposure images and reduces the displacement between adjacent two exposure images, thereby avoiding ghosting or ghosting problems when fusing the plurality of exposure images, and improving the quality of the output image obtained by fusion.

[0101] Figure 5 is a flowchart of another image processing method according to an exemplary embodiment, executed by an electronic device, see Figure 5 The method comprises the following steps:

[0102] Step S501: In response to the detection of a shooting request, obtain the exposure information of the current shooting scene.

[0103] The implementation method of step S501 is the same as that of steps S301 and S401 above, and will not be repeated here.

[0104] Step S502: Based on the exposure information, image data for each row is acquired sequentially. When the acquisition of image data for each row is completed, the image data is saved until the image data for the last row is saved.

[0105] In this embodiment of the disclosure, when acquiring an exposure image, image data for each row is acquired sequentially. A row refers to a line of pixels in the image. When the acquisition of image data for each row is completed, the image data is saved.

[0106] In some embodiments, the frame length and line count are determined. The frame length refers to the length of one line of image data being acquired, and the line count refers to the total number of lines in the exposure image to be acquired. Then, based on the exposure information, frame length, and line count, image data for each line is acquired sequentially.

[0107] Optionally, the frame length and / or number of lines are determined based on a second preset function, wherein the second preset relationship characterizes the relationship between the exposure value and the frame length and / or number of lines. Alternatively, the frame length and number of lines corresponding to the exposure value are determined based on a second correspondence relationship, wherein the second correspondence relationship includes the exposure value and the corresponding frame length and number of lines.

[0108] In some embodiments, each row of image data includes image sub-data corresponding to different exposure values. Saving the image data includes: saving each image sub-data to a corresponding data storage area, wherein the number of data storage areas is the same as the number of exposure values. That is, the image sub-data corresponding to each exposure value is stored separately. The data storage area can be a memory, a register, or other area capable of storing data.

[0109] For example, see Figure 6 The diagram shown illustrates the output method. The exposure information includes three different exposure values ​​(low exposure, medium exposure, and high exposure). Image data for each row is collected sequentially. Taking the first row as an example, image data represented by "line segment 1", "line segment 2", and "line segment 3" are collected sequentially. Low exposure corresponds to "line segment 1", medium exposure corresponds to "line segment 1" and "line segment 2", and high exposure corresponds to "line segment 1", "line segment 2", and "line segment 3". It can also be seen from the stored image data that there are more image sub-data corresponding to high exposure than to medium exposure, and more image sub-data corresponding to medium exposure than to low exposure.

[0110] Furthermore, fromFigure 6 It can also be seen that after starting to collect the image data of the first row, the image data of the next row is collected after a preset time interval, and the preset time interval is small, so as to make the time interval between the image data of multiple rows as small as possible, so as to obtain all the image data as quickly as possible.

[0111] It should be noted that in some embodiments, the image data of each row is collected in sequence based on the exposure time and the gain corresponding to each exposure value.

[0112] In step S503, the exposure image corresponding to each exposure value is determined based on the saved image data.

[0113] In the embodiments of the present disclosure, after the image data of all rows is saved, the exposure image corresponding to each exposure value is determined based on the saved image data, so that the exposure images corresponding to different exposure values are obtained at the same time.

[0114] In some embodiments, the saved image sub-data is read from each data storage area, and the exposure image corresponding to each exposure value is determined.

[0115] Optionally, the image format of the exposure image is a first format, for example, the first format is a RAW (a kind of image format) format or other formats.

[0116] In step S504, for each frame of exposure image, the exposure image is subjected to format conversion processing to obtain an exposure image in a second format.

[0117] The second format is different from the first format, for example, the second format is a YUV (a kind of color encoding method) format or other formats.

[0118] In some embodiments, referring to Figure 7As shown in the flowchart of the ISP processing, the image data collected by the sensor is stored in the memory, the exposure image in the first format is read from the memory, and then the exposure image is sequentially subjected to phase detection phase calibration (PDPC), channel gain, wide color gamut HDR, green imbalance correction, adaptive bell filtering (ABF), black level substration, lens roll off, white balance, demosaic, color correction (CC), global tone mapping (GTM), gamma, color space transfrom, hybrid noise reduction, down scaler, color correction, image processing engine processing, to obtain the output exposure image in the second format. In addition, bad pixel correction, bad pixel pair correction, grid noise reduction, and the like can also be performed.

[0119] It should be noted that, Figure 7 Only taking the execution of the above processing as an example, in another embodiment, the exposure image in the first format can be processed by using part of the above processing modes, in other processing orders, to obtain the exposure image in the second format.

[0120] In step S505, the multiple exposure images and the weight corresponding to each exposure image are fused to obtain an output image.

[0121] When fusing the multiple exposure images, the weight corresponding to each exposure image is considered, and the multiple exposure images are fused based on the weight corresponding to each exposure image to obtain an output image.

[0122] In some embodiments, after obtaining the output image, the output image can continue to be processed by a single-frame algorithm, for example, the single-frame algorithm processing includes noise reduction processing, white balance processing, filtering processing and the like, and then the output image processed by the single-frame algorithm is processed by YUV2JPEG hardware to obtain an output image in a third format, and the output image in the third format is displayed to the user. The third format can be a JPEG (Joint Photographic Experts Group) format or other formats.

[0123] In some embodiments, a weight fusion algorithm of Gaussian / Laplacian pyramid is adopted to fuse the multi-frame exposure images to obtain an output image. For example, refer to Figure 8 The schematic diagram shown in the figure, two exposure images of different exposure levels are obtained, then the Laplacian pyramid image corresponding to the exposure image is convolved with the Gaussian pyramid image, and then the two convolved images are fused to obtain an output image, and finally the output image is converted into an output image of a third format. Wherein, the Gaussian pyramid image can be regarded as the weight corresponding to each exposure image.

[0124] The method provided by the embodiments of the present disclosure can shorten the output time, solve the frame-to-frame displacement problem, and effectively solve the ghosting and ghosting problems that occur during multi-frame fusion by solving the frame-to-frame displacement problem.

[0125] Compared with the multi-frame exposure fusion scheme in the related art, the scheme provided by the embodiments of the present disclosure has higher output efficiency.

[0126] For example, refer to the comparison results shown in Table 1 below:

[0127] Table 1

[0128] Comparison item Time taken to take picture / ms Time taken by algorithm / ms Multiple frame sequential picture taking 99 500 The present scheme 33 500

[0129] Wherein, the multi-frame sequential image taking refers to the image taking manner in the first related art, the image taking time refers to the time length from starting image taking to obtaining the output image, and the algorithm time refers to the time length of ISP processing and fusion processing of the exposure image after obtaining the exposure image. The image taking efficiency of the present scheme is improved by more than 50% compared with the image taking efficiency of the multi-frame sequential image taking scheme, and the algorithm time remains the same. Compared with the multi-frame sequential image taking scheme, the present scheme improves the output efficiency.

[0130] Figure 9 is a block diagram of an image processing device according to an exemplary embodiment, configured in an electronic device, refer to Figure 9 The device comprises:

[0131] The exposure information acquisition module 901 is configured to acquire the exposure information of the current shooting scene in response to detecting a shooting request, the shooting request comprising exposure information, and the exposure information comprising a plurality of exposure values, the exposure value being used to represent the exposure degree of the exposure image.

[0132] The exposure image acquisition module 902 is configured to acquire an exposure image corresponding to each exposure value based on the exposure information.

[0133] The output image acquisition module 903 is configured to fuse the multi-frame exposure images to obtain an output image.

[0134] In some embodiments, the exposure image acquisition module 902 is configured to:

[0135] determine, based on the exposure information and a preset relationship, an exposure time and a gain corresponding to each exposure value, the preset relationship being used to represent a relationship between the exposure value and the exposure time and the gain;

[0136] acquire, based on the exposure time and the gain corresponding to each exposure value, an exposure image corresponding to each exposure value.

[0137] In some embodiments, the exposure image acquisition module 902 is configured to:

[0138] adjust the gain corresponding to each exposure value to obtain an adjusted gain corresponding to each exposure value, wherein the adjusted gains are the same;

[0139] adjust, based on an adjustment range of the gain corresponding to each exposure value, the exposure time corresponding to each exposure value to obtain an adjusted exposure time corresponding to each exposure value.

[0140] In some embodiments, the exposure image acquisition module 902 is configured to:

[0141] acquire, based on the exposure information, image data of each row in sequence, save the image data when the image data of each row is acquired, and save the image data of the last row until the image data of the last row is saved.

[0142] determine, based on the saved image data, an exposure image corresponding to each exposure value.

[0143] In some embodiments, the image data of each row includes image sub-data corresponding to different exposure values; and the exposure image acquisition module 902 is configured to:

[0144] save each image sub-data to a corresponding data storage area, the number of the data storage areas being the same as the number of the exposure values;

[0145] read the saved image sub-data from each data storage area to determine an exposure image corresponding to each exposure value.

[0146] In some embodiments, the image format of the exposure image is a first format; and the apparatus further includes:

[0147] a format conversion module configured to, for each frame of exposure image, perform format conversion processing on the exposure image to obtain an exposure image in a second format, the second format being different from the first format.

[0148] In some embodiments, the output image acquisition module 903 is configured to:

[0149] The output image is obtained by fusing the multi-exposure image and the weight corresponding to each exposure image.

[0150] In some embodiments, the apparatus further includes:

[0151] The request generation module is configured to generate a shooting request in response to detecting the shooting instruction.

[0152] As to the apparatus in the above embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0153] The embodiments of the present disclosure further provide an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the image processing method in the above embodiments.

[0154] Figure 10 is a block diagram of an electronic device 1000 according to an exemplary embodiment.

[0155] Referring to Figure 10 , the electronic device 1000 can include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0156] The processing component 1002 usually controls overall operations of the electronic device 1000, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 1002 can include one or more processors 1020 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 1002 can include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 can include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0157] The memory 1004 is configured to store various types of data to support the operations of the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1004 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0158] The power supply component 1006 supplies power for various components of the electronic device 1000. The power supply component 1006 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0159] The multimedia component 1008 includes a screen providing an output interface between the electronic device 1000 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. The front camera and / or the rear camera can receive external multimedia data when the electronic device 1000 is in an operation mode, such as a photographing mode or a video mode. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0160] The audio component 1010 is configured to output and / or input an audio signal. For example, the audio component 1010 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting an audio signal.

[0161] The I / O interface 1012 provides an interface between the processing component 1002 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0162] The sensor component 1014 includes one or more sensors for providing status assessments for various aspects of the electronic device 1000. For example, the sensor component 1014 can detect an open / closed position of the electronic device 1000, relative positioning of components, such as a display and a keypad of the electronic device 1000, a change in position of the electronic device 1000 or a component of the electronic device 1000, presence or absence of user contact with the electronic device 1000, orientation or acceleration / deceleration / g-force and temperature of the electronic device 1000. The sensor component 1014 can include an optical sensor for detecting ambient light, a proximity sensor for detecting nearby objects without any physical touch, a CMOS or CCD image sensor for use in imaging applications, and / or a gyroscope sensor, a magnetometer sensor, a pressure sensor, or a temperature sensor in some embodiments.

[0163] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0164] In an example embodiment, the electronic device 1000 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements to perform the above-described methods.

[0165] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 1004 including instructions, is also provided, which can be executed by the processor 1020 of the electronic device 1000 to implement the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0166] The embodiment of the present disclosure further provides a non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method in the above embodiment.

[0167] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0168] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is indicated only by the appended claims.

Claims

1. An image processing method, characterized by, The method comprises: in response to detecting a shooting request, acquiring exposure information of a current shooting scene, the shooting request comprising the exposure information, the exposure information comprising a plurality of exposure values for representing exposure degrees of exposure images; based on the exposure information, acquiring an exposure image corresponding to each exposure value respectively; based on a plurality of exposure images, performing fusion to obtain an output image; the exposure information, acquiring an exposure image corresponding to each exposure value respectively, comprises: based on the exposure information, sequentially acquiring image data of each row, when the image data of each row is acquired, saving the image data, until the image data of the last row is saved; based on the saved image data, determining an exposure image corresponding to each exposure value; each row of image data comprises image sub-data corresponding to different exposure values; saving the image data comprises: saving each image sub-data to a corresponding data storage area, the number of data storage areas being the same as the number of exposure values; based on the saved image data, determining an exposure image corresponding to each exposure value, comprises: reading the saved image sub-data from each data storage area to determine an exposure image corresponding to each exposure value.

2. The method of claim 1, wherein, the exposure information, acquiring an exposure image corresponding to each exposure value respectively, comprises: based on the exposure information and a preset relationship, determining an exposure time and a gain corresponding to each exposure value respectively, the preset relationship representing a relationship between exposure values, exposure times and gains; based on the exposure time and the gain corresponding to each exposure value respectively, acquiring an exposure image corresponding to each exposure value.

3. The method of claim 2, wherein, before the exposure time and the gain corresponding to each exposure value are acquired, the method further comprises: adjusting the gain corresponding to each exposure value respectively to obtain an adjusted gain corresponding to each exposure value, wherein the adjusted gains are the same; based on the adjustment range of the gain corresponding to each exposure value, adjusting the exposure time corresponding to each exposure value to obtain an adjusted exposure time corresponding to each exposure value.

4. The method of claim 1, wherein, The image format of the exposure image is a first format; after the exposure image corresponding to each exposure value is acquired, the method further comprises: for each exposure image, performing format conversion processing on the exposure image to obtain an exposure image of a second format, the second format being different from the first format.

5. The method of claim 1, wherein, based on a plurality of exposure images and a weight corresponding to each exposure image, performing fusion to obtain the output image. before the exposure information of the current shooting scene is acquired in response to detecting the shooting request, the method further comprises:

6. The method of claim 1, wherein, in response to detecting a shooting instruction, generating the shooting request. the device comprises:

7. An image processing apparatus characterized by comprising: ​ An exposure information acquisition module is configured to acquire exposure information of a current shooting scene in response to detecting a shooting request, the shooting request including the exposure information, the exposure information including a plurality of exposure values used to represent exposure degrees of exposure images; An exposure image acquisition module is configured to acquire an exposure image corresponding to each of the exposure values based on the exposure information; An output image acquisition module is configured to obtain an output image by fusing a plurality of the exposure images; The exposure image acquisition module is further configured to: acquire image data of each row based on the exposure information, and save the image data when the image data of each row is acquired, until the image data of the last row is saved; determine an exposure image corresponding to each of the exposure values based on the saved image data; The image data of each row includes image sub-data corresponding to different exposure values; the exposure image acquisition module is further configured to: save each of the image sub-data to a corresponding data storage area, the number of the data storage areas being the same as the number of the exposure values; read the saved image sub-data from each of the data storage areas to determine an exposure image corresponding to each of the exposure values.

8. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can perform the method of any one of claims 1-6.

Images