Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium. The method comprises the steps that exposure information of a current shooting scene is acquired in response to a detected shooting request, the shooting request comprises the exposure information, the exposure information comprises a plurality of exposure values, and the exposure values are used for representing the exposure degree of an exposure image; respectively acquiring an exposure image corresponding to each exposure value based on the exposure information; and performing fusion based on the multiple frames of exposure images to obtain an output image. According to the image output mode, the exposure images with different exposure degrees are requested to be shot according to one shooting request, the multiple frames of exposure images are obtained at a time, the time difference between the two adjacent frames of exposure images is reduced, the displacement between the two adjacent frames of exposure images is reduced, the problem of ghosting or ghosting during fusion of the multiple frames of exposure images is avoided, and the fusion efficiency of the multiple frames of exposure images is improved. And the quality of the fused output image is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the widespread use of various electronic devices, users are using the camera function of electronic devices more and more, and their requirements for image quality are getting higher and higher. As a common camera function, High Dynamic Range (HDR) technology can provide users with high-contrast effect images, restore dynamic range scenes, improve image quality, and bring users a very good image quality experience. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an image processing method, device, electronic device and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, the method comprising:

[0005] In response to detecting a shooting request, acquiring exposure information of a current shooting scene, the shooting request including the exposure information, the exposure information including a plurality of exposure values, the exposure values ​​being used to represent an exposure degree of an exposed image;

[0006] Based on the exposure information, respectively obtain exposure images corresponding to each exposure value;

[0007] The exposure images are fused based on multiple frames to obtain an output image.

[0008] In some embodiments, the acquiring, based on the exposure information, exposure images corresponding to each exposure value respectively includes:

[0009] Based on the exposure information and a preset relationship, respectively determine the exposure time and gain corresponding to each of the exposure values, wherein the preset relationship is used to characterize the relationship between the exposure value and the exposure time and gain;

[0010] Based on the exposure time and gain corresponding to each exposure value, an exposure image corresponding to each exposure value is acquired respectively.

[0011] In some embodiments, before acquiring the exposure images corresponding to each exposure value based on the exposure time and gain corresponding to each exposure value, the method further includes:

[0012] respectively adjusting the gain corresponding to each of the exposure values ​​to obtain an adjusted gain corresponding to each of the exposure values, wherein the adjusted multiple gains are the same;

[0013] Based on the adjustment amplitude of the gain corresponding to each of the exposure values, the exposure time corresponding to each of the exposure values ​​is adjusted to obtain the adjusted exposure time corresponding to each of the exposure values.

[0014] In some embodiments, the acquiring, based on the exposure information, exposure images corresponding to each exposure value respectively includes:

[0015] Based on the exposure information, sequentially collect image data of each row, and when the image data of each row is collected, save the image data until the image data of the last row is saved;

[0016] Based on the stored image data, an exposure image corresponding to each exposure value is determined.

[0017] In some embodiments, each row of image data includes image sub-data corresponding to different exposure values; and saving the image data includes:

[0018] respectively saving each of the image sub-data to a corresponding data storage area, wherein the number of the data storage areas is the same as the number of the exposure values;

[0019] The step of determining, based on the stored image data, an exposure image corresponding to each exposure value comprises:

[0020] The stored image sub-data are read from each of the data storage areas to determine an exposure image corresponding to each of the exposure values.

[0021] In some embodiments, the image format of the exposure image is a first format; after acquiring the exposure image corresponding to each exposure value based on the exposure information, the method further includes:

[0022] For each frame of the exposure image, a format conversion process is performed on the exposure image to obtain an exposure image in a second format, where the second format is different from the first format.

[0023] In some embodiments, the step of fusing the multiple exposure images to obtain an output image includes:

[0024] The output image is obtained by fusing the multiple frames of exposure images and the weights corresponding to each frame of the exposure image.

[0025] In some embodiments, before acquiring exposure information of a current shooting scene in response to detecting a shooting request, the method further includes:

[0026] In response to detecting a shooting instruction, the shooting request is generated.

[0027] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:

[0028] an exposure information acquisition module, configured to acquire exposure information of a current shooting scene in response to detecting a shooting request, wherein the shooting request includes the exposure information, and the exposure information includes a plurality of exposure values, and the exposure values ​​are used to represent the exposure degree of the exposure image;

[0029] An exposure image acquisition module is configured to respectively acquire an exposure image corresponding to each exposure value based on the exposure information;

[0030] The output image acquisition module is configured to fuse multiple frames of exposure images to obtain an output image.

[0031] In some embodiments, the exposure image acquisition module is configured to:

[0032] Based on the exposure information and a preset relationship, respectively determine the exposure time and gain corresponding to each of the exposure values, wherein the preset relationship is used to characterize the relationship between the exposure value and the exposure time and gain;

[0033] Based on the exposure time and gain corresponding to each exposure value, an exposure image corresponding to each exposure value is acquired respectively.

[0034] In some embodiments, the exposure image acquisition module is further configured to:

[0035] respectively adjusting the gain corresponding to each of the exposure values ​​to obtain an adjusted gain corresponding to each of the exposure values, wherein the adjusted multiple gains are the same;

[0036] Based on the adjustment amplitude of the gain corresponding to each of the exposure values, the exposure time corresponding to each of the exposure values ​​is adjusted to obtain the adjusted exposure time corresponding to each of the exposure values.

[0037] In some embodiments, the exposure image acquisition module is configured to:

[0038] Based on the exposure information, sequentially collect image data of each row, and when the image data of each row is collected, save the image data until the image data of the last row is saved;

[0039] Based on the stored image data, an exposure image corresponding to each exposure value is determined.

[0040] In some embodiments, the image data of each row includes image sub-data corresponding to different exposure values; and the exposure image acquisition module is configured to:

[0041] respectively saving each of the image sub-data to a corresponding data storage area, wherein the number of the data storage areas is the same as the number of the exposure values;

[0042] The stored image sub-data are read from each of the data storage areas to determine an exposure image corresponding to each of the exposure values.

[0043] In some embodiments, the image format of the exposure image is a first format; and the apparatus further comprises:

[0044] The format conversion module is configured to perform format conversion processing on each frame of the exposure image to obtain an exposure image in a second format, where the second format is different from the first format.

[0045] In some embodiments, the output image acquisition module is configured to:

[0046] The output image is obtained by fusing the multiple frames of exposure images and the weights corresponding to each frame of the exposure image.

[0047] In some embodiments, the apparatus further comprises:

[0048] The request generating module is configured to generate the shooting request in response to detecting a shooting instruction.

[0049] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0050] processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is configured to execute the method as described in the first aspect of the embodiment of the present disclosure.

[0053] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in the first aspect of the embodiment of the present disclosure.

[0054] The above method of the present disclosure has the following beneficial effects:

[0055] The method provided by the embodiment of the present disclosure obtains exposure information of the current shooting scene in response to detecting a shooting request, the shooting request includes exposure information, the exposure information includes multiple exposure values, and the exposure value is used to characterize the exposure degree of the exposure image; based on the exposure information, the exposure image corresponding to each exposure value is obtained respectively; based on the multiple frames of exposure images, the output image is obtained by fusion. This output method requests to shoot multiple exposure images with different exposure degrees according to a shooting request, and obtains multiple frames of exposure images at one time, which reduces the time difference between two adjacent frames of exposure images and reduces the displacement between two adjacent frames of exposure images, thereby avoiding ghosting or double image problems when multiple frames of exposure images are fused, and improving the quality of the fused output image.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1 is a schematic diagram showing a method of outputting a graph in a related technology according to an exemplary embodiment;

[0059] Figure 2 is a schematic diagram showing another method of outputting a graph in a related technology according to an exemplary embodiment;

[0060] Figure 3 is a flow chart of an image processing method according to an exemplary embodiment;

[0061] Figure 4 is a flow chart showing another image processing method according to an exemplary embodiment;

[0062] Figure 5 is a flow chart showing another image processing method according to an exemplary embodiment;

[0063] Figure 6 is a schematic diagram showing a method of outputting a graph according to an exemplary embodiment;

[0064] Figure 7 is a flow chart showing an ISP process according to an exemplary embodiment;

[0065] Figure 8 is a schematic diagram showing a fused exposure image according to an exemplary embodiment;

[0066] Fig. 9is a block diagram of an image processing device according to an exemplary embodiment;

[0067] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0069] HDR usually uses multiple frames of images with different exposure levels to fuse a high dynamic range image. HDR technology needs to provide a good image quality experience in different scenes, which is also a very big challenge. For example, when there is a moving object in the shooting scene, the user wants to take a picture of the current moving object, but the moving object may move too much or not be in the image, resulting in a poor user experience.

[0070] In a related technology, an exposure image is outputted in a frame-by-frame manner: multiple shooting requests are received, and a different exposure image is outputted according to each shooting request. Figure 1 , Figure 1 The horizontal axis represents time, and the vertical axis represents the number of rows. The number of rows refers to the number of rows of pixels in the image. When outputting images, the exposure images are output frame by frame. N, N+1, N+2, etc. represent 1 frame of exposure image respectively. In this output method, the time difference between two adjacent exposure frames is the difference between the exposure start time points of the two exposure frames.

[0071] In another related technology, an exposure image is outputted in a row-by-row output mode: a shooting request is received, and different exposure images are outputted according to the shooting request, wherein the shooting request carries multiple request information, and each request information is used to request the output of a frame of exposure image. Figure 2 , Figure 2 The horizontal axis represents time, and the vertical axis represents the number of rows. When outputting images, multiple exposure frames are output simultaneously. Figure 2 The example above takes the simultaneous output of three exposure images, such as the simultaneous output of exposure images N, N+1, and N+2. In this output method, the time difference between two adjacent exposure images is the difference generated when switching between rows, that is, row blanking (HBlank). This output method can also be called Staggered HDR (single-frame progressive HDR).

[0072] In the above two related technologies, when acquiring multiple frames of exposure images, there will be a long time difference between two adjacent frames of exposure images, that is, the displacement between the two adjacent frames of exposure images is large, which leads to ghosting or double imaging problems when the multiple frames of exposure images are fused, resulting in poor quality of the fused output image.

[0073] The disclosed embodiment proposes a new image processing method, which can reduce the time difference between two adjacent frames of exposure images, that is, reduce the displacement between two adjacent frames of exposure images, thereby avoiding ghosting or double imaging problems when multiple frames of exposure images are fused, and improving the quality of the fused output image.

[0074] The method provided in the embodiment of the present disclosure is executed by an electronic device, which may be an electronic device with a camera function, such as a mobile phone, a tablet computer, a laptop computer, or a camera device.

[0075] Figure 3 is a flowchart of an image processing method according to an exemplary embodiment, which is executed by an electronic device, see Figure 3 , the method comprises the following steps:

[0076] Step S301, in response to detecting a shooting request, acquiring exposure information of a current shooting scene, the shooting request including exposure information, the exposure information including a plurality of exposure values, and the exposure values ​​are used to represent the exposure degree of an exposed image.

[0077] Among them, the shooting request is used to request the shooting of multiple exposure images, and the exposure information includes multiple exposure values ​​(ExposureValue, EV), which is used to characterize the exposure degree of the exposure image. For example, the exposure information includes 3 exposure values ​​EV0, EV- and EV+, EV0 represents medium exposure (normal exposure), EV- represents low exposure (dark), and EV+ represents high exposure (bright).

[0078] Since the shooting request includes exposure information, in response to detecting the shooting request, the exposure information of the current shooting scene can be acquired from the shooting request, wherein the current shooting scene is the picture currently being shot.

[0079] Step S302: based on the exposure information, respectively obtain an exposure image corresponding to each exposure value.

[0080] Since the exposure information includes multiple exposure values, a corresponding exposure image can be obtained according to each exposure value. When acquiring, the exposure image corresponding to each exposure value is acquired according to the exposure degree represented by the exposure value, and the exposure images corresponding to different exposure values ​​show different exposure degrees. In addition, when acquiring the exposure image, it can be acquired according to multiple exposure values ​​at the same time, thereby reducing the time difference when different exposure images are acquired and reducing the displacement between two adjacent frames of exposure images.

[0081] Step S303: fusing multiple exposure images to obtain an output image.

[0082] Fusion is performed based on multiple exposure images, that is, exposure images at different exposure levels are comprehensively considered to obtain an output image, which integrates image information at different exposure levels. The output image has better image quality and contains more image information than a single exposure image.

[0083] The method provided by the embodiment of the present disclosure obtains exposure information of the current shooting scene in response to detecting a shooting request, the shooting request includes exposure information, the exposure information includes multiple exposure values, and the exposure value is used to characterize the exposure degree of the exposure image; based on the exposure information, the exposure image corresponding to each exposure value is obtained respectively; based on the multiple frames of exposure images, the output image is obtained by fusion. This output method requests to shoot exposure images with different exposure degrees according to a shooting request, and obtains multiple frames of exposure images at one time, which reduces the time difference between two adjacent frames of exposure images and reduces the displacement between two adjacent frames of exposure images, thereby avoiding ghosting or double image problems when multiple frames of exposure images are fused, and improving the quality of the fused output image.

[0084] The image processing method provided by the embodiment of the present disclosure includes multi-frame request, multi-frame image acquisition, ISP (Image Signal Processor, image signal processor) processing, multi-frame algorithm fusion and output Figure 5 These stages are described respectively through the following embodiments.

[0085] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment, which is executed by an electronic device, see Figure 4 , the method comprises the following steps:

[0086] Step S401, in response to detecting a shooting request, acquiring exposure information of a current shooting scene.

[0087] The shooting request includes exposure information, and the exposure information includes multiple exposure values. The exposure values ​​are used to represent the exposure degree of the exposed image.

[0088] In some embodiments, in response to detecting a shooting instruction, a shooting request is generated. For example, a user clicks a shooting button, and the electronic device detects the user's clicking operation on the shooting button, determines that a shooting instruction is detected, and generates a shooting request.

[0089] Optionally, the exposure value may be randomly set or calculated using a preset algorithm. For example, the electronic device is configured with a preset algorithm, and during the image preview process, the preset algorithm is used to calculate the exposure information based on the current ambient brightness, color temperature, sensitivity, gain, current shooting scene, and other information. Of course, the exposure value may also be determined in other ways, and the embodiments of the present disclosure are not limited thereto.

[0090] Step S402: Based on the exposure information and the preset relationship, the exposure time and gain corresponding to each exposure value are determined respectively.

[0091] Among them, when actually collecting the exposure image, it is necessary to collect it according to the exposure time (ExposureTime) and gain. The exposure time refers to the time interval from the opening to the closing of the shutter. In the case of no overexposure, increasing the exposure time can increase the signal-to-noise ratio and make the image clearer; the gain refers to the amplification gain of the analog signal after double sampling.

[0092] The preset relationship is used to characterize the relationship between the exposure value and the exposure time and gain. Optionally, the preset relationship is a first preset function, and the exposure time and gain are determined according to the exposure value and the first preset function; or, the preset relationship is a preset first corresponding relationship, and the first corresponding relationship includes the exposure value and the corresponding exposure time and gain.

[0093] In some embodiments, the electronic device includes an automatic exposure (AE) module, which is used to convert the exposure value to obtain the exposure time and gain corresponding to the exposure value. For example, the exposure time and gain after the exposure information (EV0, EV-10, EV+8) is converted to: ExposureTime:Gain (10000000:1.2, 8000000:1.0, 15000000:2.0). The unit of exposure time is nanoseconds.

[0094] Optionally, in order to facilitate the subsequent acquisition of exposure images, the gains corresponding to different exposure values ​​can be made the same, that is, the gains corresponding to each exposure value are adjusted respectively to obtain the adjusted gains corresponding to each exposure value, wherein the multiple gains after adjustment are the same; based on the adjustment amplitude of the gain corresponding to each exposure value, the exposure time corresponding to each exposure value is adjusted to obtain the adjusted exposure time corresponding to each exposure value. In other words, the gains corresponding to each exposure value are adjusted to the same value, and the exposure time corresponding to each exposure value is adjusted according to the adjustment amplitude of the corresponding gain, so that the exposure degree represented by the adjusted set of exposure time and gain is the same as before the adjustment.

[0095] Optionally, based on the adjustment amplitude of the gain corresponding to each exposure value, the exposure time corresponding to each exposure value is adjusted, including: for each exposure value, the exposure time corresponding to the exposure value is set to the corresponding adjustment amplitude of the gain to obtain the adjusted exposure time.

[0096] For example, adjust the exposure time and gain: ExposureTime:Gain(10000000:1.2, 8000000:1.0, 15000000:2.0) to: ExposureTime:Gain(12000000:1.0, 8000000:1.0, 30000000:1.0), and the adjusted gains are all 1.

[0097] Step S403: acquiring exposure images corresponding to each exposure value based on the exposure time and gain corresponding to each exposure value.

[0098] The exposure time and gain corresponding to each exposure value may be before or after adjustment.

[0099] Step S404: fusing multiple exposure images to obtain an output image.

[0100] The method provided by the embodiment of the present disclosure converts the exposure value into exposure time and gain, and collects the exposure image based on the exposure time and gain. This output method requests to shoot exposure images with different exposure levels according to a shooting request, and obtains multiple frames of exposure images at one time, thereby reducing the time difference between two adjacent frames of exposure images and reducing the displacement between two adjacent frames of exposure images, thereby avoiding ghosting or double imaging problems when multiple frames of exposure images are fused, and improving the quality of the fused output image.

[0101] Figure 5 is a flowchart of another image processing method according to an exemplary embodiment, which is executed by an electronic device, see Figure 5 , the method comprises the following steps:

[0102] Step S501 , in response to detecting a shooting request, acquiring exposure information of a current shooting scene.

[0103] The implementation of step S501 refers to the above steps S301 and S401, which will not be described in detail here.

[0104] Step S502 , based on the exposure information, sequentially collect image data of each row, and when the collection of image data of each row is completed, save the image data until the image data of the last row is saved.

[0105] In the embodiment of the present disclosure, when collecting the exposure image, the image data of each row is collected in sequence, and a row refers to a row composed of pixel points in the image. When the collection of the image data of each row is completed, the image data is saved.

[0106] In some embodiments, the frame length (FrameLength) and the number of lines (Linecount) are determined, where the frame length refers to the length of a line of image data collected, and the number of lines refers to the total number of lines in the exposure image to be collected. Then, based on the exposure information, the frame length and the number of lines, the image data of each line is collected in sequence.

[0107] Optionally, according to a second preset function, according to the exposure value and the second preset function, the frame length and / or the number of rows are determined, and the second preset relationship is used to characterize the relationship between the exposure value and the frame length and / or the number of rows. Alternatively, according to a second corresponding relationship, the frame length and the number of rows corresponding to the exposure value are determined, and the second corresponding relationship includes the exposure value and the corresponding frame length and the number of rows.

[0108] In some embodiments, each row of image data includes image sub-data corresponding to different exposure values, and saving the image data includes: saving each image sub-data to a corresponding data storage area, and the number of data storage areas is the same as the number of exposure values. In other words, the image sub-data corresponding to each exposure value is stored respectively. The data storage area may be a memory, a register, or other area that can store data.

[0109] For example, see Figure 6 The schematic diagram of the image output method shown in the figure includes three different exposure values ​​(i.e., low exposure, medium exposure and high exposure). The image data of each row is collected in turn. Taking the first row as an example, the image data represented by "line segment 1", "line segment 2" and "line segment 3" are collected in turn, among which, low exposure corresponds to "line segment 1", medium exposure corresponds to "line segment 1" and "line segment 2", and high exposure corresponds to "line segment 1", "line segment 2" and "line segment 3". It can also be seen from the stored image data that the image sub-data corresponding to the high exposure is more than the image sub-data corresponding to the medium exposure, and the image sub-data corresponding to the medium exposure is more than the image sub-data corresponding to the low exposure.

[0110] And, from Figure 6 It can also be seen that after the first row of image data is collected, the next row of image data is collected at a preset time interval, and the preset time interval is small, so that the time interval between multiple rows of image data is as small as possible to obtain all the image data faster.

[0111] It should be noted that, in some embodiments, based on the exposure time and gain corresponding to each exposure value, the image data of each row is collected in sequence.

[0112] Step S503: determining an exposure image corresponding to each exposure value based on the stored image data.

[0113] In the disclosed embodiment, after the image data of all rows are saved, the exposure image corresponding to each exposure value is determined based on the saved image data, so that exposure images corresponding to different exposure values ​​are obtained at the same time.

[0114] In some embodiments, the stored image sub-data is read from each data storage area to determine the exposure image corresponding to each exposure value.

[0115] Optionally, the image format of the exposure image is a first format, for example, the first format is a RAW (an image format) format or other formats.

[0116] Step S504: for each frame of exposure image, perform format conversion processing on the exposure image to obtain an exposure image in a second format.

[0117] The second format is different from the first format, for example, the second format is a YUV (a color encoding method) format or other formats.

[0118] In some embodiments, see Figure 7In the flowchart of ISP processing shown in the figure, the image data collected by the sensor is stored in the memory, the exposure image of the first format is read from the memory, and then the exposure image is subjected to phase calibration (PDPC), channel gain (Channels Gains), wide color gamut HDR, green imbalance correction (Green imbalance correction), adaptive bell filtering (ABF), black level substration, lens roll off, white balance, demosaic, color correction (CC), global tone mapping (GTM), overall brightening (Gamma), color space conversion (color spacetransfrom), hybrid noise reduction (hybrid noise reduction), downsampling (Down Scaler), color correction, and image processing engine processing in sequence to obtain the output exposure image of the second format. In addition, bad pixel correction, bad pixel pair correction, grid noise reduction and other processing can also be performed.

[0119] It should be noted that Figure 7 Taking the execution of the above processing as an example, in another embodiment, some of the above processing methods may be used to process the exposure image in the first format in other processing orders to obtain the exposure image in the second format.

[0120] Step S505 , fusing the multiple exposure images and the weight corresponding to each exposure image to obtain an output image.

[0121] When fusing multiple frames of exposure images, the weight corresponding to each frame of exposure image is considered, and the multiple frames of exposure images are fused based on the weight corresponding to each frame of exposure image to obtain an output image.

[0122] In some embodiments, after obtaining the output image, the output image may continue to be subjected to single-frame algorithm processing, for example, the single-frame algorithm processing includes noise reduction processing, white balance processing, filtering processing, and the like, and then the output image after the single-frame algorithm processing is processed by YUV2JPEG hardware to obtain an output image in a third format, and the output image in the third format is displayed to the user. The third format may be a JPEG (Joint Photographic Experts Group) format or other formats.

[0123] In some embodiments, a Gaussian / Laplacian pyramid weight fusion algorithm is used to fuse multiple exposure images to obtain an output image. Figure 8 As shown in the schematic diagram, two exposure images with different exposure levels are obtained, and then the Laplacian pyramid image corresponding to the exposure image is convolved with the Gaussian pyramid image, and then the two convolved images are fused to obtain an output image, and finally the output image is converted into an output image of the third format. Among them, the Gaussian pyramid image can be regarded as the weight corresponding to each exposure image.

[0124] The method provided by the embodiment of the present disclosure can shorten the image output time, solve the problem of displacement between frames, and effectively solve the ghost and double image problems that occur when multiple frames are merged by solving the problem of displacement between frames.

[0125] Compared with the multi-frame exposure fusion solution in the related art, the solution provided by the embodiment of the present disclosure has higher image output efficiency.

[0126] For example, see the comparison results shown in Table 1 below:

[0127] Table 1

[0128] Comparison Items Image acquisition time / ms Algorithm time / ms Multiple frames sequentially captured 99 500 This program 33 500

[0129] Among them, multi-frame sequential image acquisition refers to the image acquisition method in the first related technology mentioned above, image acquisition time refers to the time from the start of image acquisition to the output image, and algorithm time refers to the time for ISP processing and fusion processing of the exposure image after the exposure image is obtained. The image acquisition efficiency of this solution is improved by more than 50% compared with the image acquisition efficiency of the multi-frame sequential image acquisition solution, and the algorithm time remains the same. Compared with the multi-frame sequential image acquisition solution, this solution improves the image output efficiency.

[0130] Fig. 9 is a block diagram of an image processing device according to an exemplary embodiment, which is configured in an electronic device, see Fig. 9 , the device comprises:

[0131] The exposure information acquisition module 901 is configured to acquire exposure information of a current shooting scene in response to detecting a shooting request, wherein the shooting request includes exposure information, and the exposure information includes a plurality of exposure values, and the exposure values ​​are used to represent the exposure degree of the exposed image;

[0132] The exposure image acquisition module 902 is configured to respectively acquire the exposure image corresponding to each exposure value based on the exposure information;

[0133] The output image acquisition module 903 is configured to perform fusion based on multiple frames of exposure images to obtain an output image.

[0134] In some embodiments, the exposure image acquisition module 902 is configured to:

[0135] Based on the exposure information and the preset relationship, respectively determine the exposure time and gain corresponding to each exposure value, where the preset relationship is used to characterize the relationship between the exposure value and the exposure time and gain;

[0136] Based on the exposure time and gain corresponding to each exposure value, an exposure image corresponding to each exposure value is acquired respectively.

[0137] In some embodiments, the exposure image acquisition module 902 is configured to:

[0138] Adjusting the gain corresponding to each exposure value respectively to obtain an adjusted gain corresponding to each exposure value, wherein the adjusted multiple gains are the same;

[0139] Based on the adjustment amplitude of the gain corresponding to each exposure value, the exposure time corresponding to each exposure value is adjusted to obtain the adjusted exposure time corresponding to each exposure value.

[0140] In some embodiments, the exposure image acquisition module 902 is configured to:

[0141] Based on the exposure information, the image data of each row is collected in sequence, and when the image data of each row is collected, the image data is saved until the image data of the last row is saved;

[0142] Based on the stored image data, an exposure image corresponding to each exposure value is determined.

[0143] In some embodiments, each row of image data includes image sub-data corresponding to different exposure values; the exposure image acquisition module 902 is configured to:

[0144] Each image sub-data is saved to a corresponding data storage area respectively, and the number of the data storage areas is the same as the number of exposure values;

[0145] The stored image sub-data is read from each data storage area to determine the exposure image corresponding to each exposure value.

[0146] In some embodiments, the image format of the exposure image is a first format; the apparatus further comprises:

[0147] The format conversion module is configured to perform format conversion processing on each frame of exposure image to obtain an exposure image in a second format, where the second format is different from the first format.

[0148] In some embodiments, the output image acquisition module 903 is configured to:

[0149] The output image is obtained by fusing multiple exposure images and the weights corresponding to each exposure image.

[0150] In some embodiments, the apparatus further comprises:

[0151] The request generating module is configured to generate a shooting request in response to detecting a shooting instruction.

[0152] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0153] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the image processing method in the above embodiment.

[0154] Fig.10 is a block diagram of an electronic device 1000 according to an exemplary embodiment.

[0155] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0156] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0157] The memory 1004 is configured to store various types of data to support operations on the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0158] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1000.

[0159] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0160] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0161] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0162] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0163] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0164] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0165] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0166] The embodiments of the present disclosure also provide a non-temporary computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image processing method in the above embodiments.

[0167] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0168] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: In response to detecting a shooting request, acquiring exposure information of a current shooting scene, the shooting request including the exposure information, the exposure information including a plurality of exposure values, the exposure values ​​being used to represent an exposure degree of an exposed image; Based on the exposure information, respectively acquiring exposure images corresponding to each exposure value; The exposure images are fused based on multiple frames to obtain an output image.

2. The method according to claim 1, characterized in that The acquiring, based on the exposure information, exposure images corresponding to each exposure value respectively includes: Based on the exposure information and a preset relationship, respectively determine the exposure time and gain corresponding to each of the exposure values, wherein the preset relationship is used to characterize the relationship between the exposure value and the exposure time and gain; Based on the exposure time and gain corresponding to each exposure value, an exposure image corresponding to each exposure value is acquired respectively.

3. The method according to claim 2, characterized in that Before respectively acquiring the exposure image corresponding to each exposure value based on the exposure time and gain corresponding to each exposure value, the method further includes: respectively adjusting the gain corresponding to each of the exposure values ​​to obtain an adjusted gain corresponding to each of the exposure values, wherein the adjusted multiple gains are the same; Based on the adjustment amplitude of the gain corresponding to each of the exposure values, the exposure time corresponding to each of the exposure values ​​is adjusted to obtain the adjusted exposure time corresponding to each of the exposure values.

4. The method according to claim 1, characterized in that The acquiring, based on the exposure information, exposure images corresponding to each exposure value respectively includes: Based on the exposure information, sequentially collect image data of each row, and when the image data of each row is collected, save the image data until the image data of the last row is saved; Based on the stored image data, an exposure image corresponding to each exposure value is determined.

5. The method according to claim 4, characterized in that The image data of each row includes image sub-data corresponding to different exposure values; and the storing of the image data includes: respectively saving each of the image sub-data to a corresponding data storage area, wherein the number of the data storage areas is the same as the number of the exposure values; The step of determining, based on the stored image data, an exposure image corresponding to each exposure value comprises: The stored image sub-data are read from each of the data storage areas to determine an exposure image corresponding to each of the exposure values.

6. The method according to claim 1, characterized in that The image format of the exposure image is a first format; after acquiring the exposure image corresponding to each exposure value based on the exposure information, the method further includes: For each frame of the exposure image, a format conversion process is performed on the exposure image to obtain an exposure image in a second format, where the second format is different from the first format.

7. The method according to claim 1, characterized in that The step of fusing the multiple exposure images to obtain an output image includes: The output image is obtained by fusing the multiple frames of exposure images and the weights corresponding to each frame of the exposure image.

8. The method according to claim 1, characterized in that Before acquiring exposure information of the current shooting scene in response to detecting the shooting request, the method further includes: In response to detecting a shooting instruction, the shooting request is generated.

9. An image processing device, characterized in that: The device comprises: an exposure information acquisition module, configured to acquire exposure information of a current shooting scene in response to detecting a shooting request, wherein the shooting request includes the exposure information, and the exposure information includes a plurality of exposure values, and the exposure values ​​are used to represent the exposure degree of the exposure image; An exposure image acquisition module, configured to respectively acquire an exposure image corresponding to each exposure value based on the exposure information; The output image acquisition module is configured to fuse multiple frames of exposure images to obtain an output image.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method as claimed in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 8.

Images