Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, an electronic device and a storage medium. The method is applied to an electronic device and includes: in response to a received image acquisition request, acquiring multiple frames of acquisition images; performing image processing on the multiple frames of acquisition images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail; and sending the thumbnail to a camera application layer of the electronic device. Through the method, the efficiency of obtaining the photographed image and the thumbnail is improved, the photographing response speed is improved, and the photographing experience is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Currently, the photographing speed of electronic equipment such as mobile phones and tablet computers is getting faster. No matter what operating system is integrated, the photographing speed of electronic equipment produced by various manufacturers is very fast. However, with the continuous development of photographing technology, people's requirements for the photographing of electronic equipment are increasing.

[0003] There are many ways to improve the photographing speed at present. From the user's point of view, as long as the gallery is refreshed after photographing, it is considered that the current photographing has been completed and the next photographing can be performed. As for the processing of the photographed image, it can be put into the post-processing queue for processing, so many manufacturers of electronic equipment use thumbnails to refresh the gallery. However, in the related technology, the thumbnail refreshing process is relatively slow, and how to improve the speed of determining the thumbnail has become a problem to be solved. SUMMARY

[0004] The present disclosure provides an image processing method and device, electronic equipment and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, applied to an electronic device, and the method comprises:

[0006] In response to a received image acquisition request, a plurality of acquisition images are acquired;

[0007] The plurality of acquisition images are processed by a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail;

[0008] The thumbnail is sent to a camera application layer of the electronic device.

[0009] In some embodiments, the processing of the plurality of acquisition images by the camera hardware layer of the electronic device to obtain the photographed image and the thumbnail comprises:

[0010] The plurality of acquisition images are fused by the camera hardware layer of the electronic device to obtain an initial image after fusion processing;

[0011] The initial image is processed to obtain the photographed image and the thumbnail.

[0012] In some embodiments, the processing of the initial image to obtain the photographed image and the thumbnail comprises:

[0013] The initial image is processed to obtain a target image;

[0014] obtaining the photograph image and the thumbnail based on the target image.

[0015] In some embodiments, the obtaining the photograph image and the thumbnail based on the target image comprises:

[0016] performing size reduction processing on the target image to obtain the thumbnail;

[0017] performing encoding processing on the target image to obtain the photograph image.

[0018] In some embodiments, the performing encoding processing on the target image to obtain the photograph image comprises:

[0019] in a case where the size of the target image is inconsistent with a preset photograph size, performing size adjustment on the target image, and performing encoding on the adjusted image to obtain the photograph image.

[0020] In some embodiments, the performing image optimization processing on the initial image to obtain the target image comprises:

[0021] performing gamut conversion on the initial image to obtain a converted image;

[0022] inputting the converted image into at least one image optimization model to obtain the target image.

[0023] In some embodiments, the method further comprises:

[0024] obtaining a current ambient brightness of an environment in which the electronic device is located;

[0025] in a case where the current ambient brightness is less than a brightness threshold, obtaining a first frame number of the plurality of frames of the acquisition image;

[0026] in a case where the current ambient brightness is greater than or equal to the brightness threshold, obtaining a second frame number of the plurality of frames of the acquisition image;

[0027] the first frame number is greater than the second frame number.

[0028] According to a second aspect of the embodiments of the present disclosure, an image processing apparatus is provided, which is applied in an electronic device, and the apparatus comprises:

[0029] a first obtaining module configured to obtain a plurality of frames of acquisition image in response to a received image acquisition request;

[0030] an image obtaining module configured to obtain a photograph image and a thumbnail by performing image processing on the plurality of frames of acquisition image through a camera hardware layer of the electronic device.

[0031] The sending module is configured to send the thumbnail to a camera application layer of the electronic device.

[0032] In some embodiments, the image obtaining module is further configured to perform fusion processing on the multiple frames of captured images through a camera hardware layer of the electronic device to obtain an initial image after fusion processing, and perform image processing on the initial image to obtain the photographed image and the thumbnail.

[0033] In some embodiments, the image obtaining module is further configured to perform image optimization processing on the initial image to obtain a target image, and obtain the photographed image and the thumbnail based on the target image.

[0034] In some embodiments, the image obtaining module is further configured to perform size reduction processing on the target image to obtain the thumbnail, and perform encoding processing on the target image to obtain the photographed image.

[0035] In some embodiments, the image obtaining module is further configured to perform size adjustment on the target image in a case where a size of the target image is inconsistent with a preset photographed size, and perform encoding on the adjusted image to obtain the photographed image.

[0036] In some embodiments, the image obtaining module is further configured to perform color gamut conversion on the initial image to obtain a converted image, and input the converted image into at least one image optimization model to obtain the target image.

[0037] In some embodiments, the apparatus further includes:

[0038] The second obtaining module is configured to obtain a current ambient brightness of an environment in which the electronic device is located.

[0039] The third obtaining module is configured to obtain a first frame number of the captured images in a case where the current ambient brightness is less than a brightness threshold, and obtain a second frame number of the captured images in a case where the current ambient brightness is greater than or equal to the brightness threshold; the first frame number is greater than the second frame number.

[0040] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0041] a processor;

[0042] a memory for storing processor-executable instructions;

[0043] The processor is configured to perform the image processing method as described in the first aspect.

[0044] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, comprising:

[0045] When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can perform the image processing method as described in the first aspect above.

[0046] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0047] In the embodiments of the present disclosure, the electronic device acquires multiple frames of captured images in response to a received image capture request; performs image processing on the multiple frames of captured images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail; and sends the thumbnail to a camera application layer of the electronic device. That is, the camera hardware layer can directly perform image processing on the multiple frames of captured images according to the embodiments of the present disclosure, without the need for the camera application layer to call data of the multiple frames of captured images for image processing. In this way, the speed of image processing on the multiple frames of captured images can be improved, and the efficiency of obtaining the photographed image and the thumbnail can be improved. Moreover, the camera application layer can be directly sent the thumbnail to refresh the image after obtaining the thumbnail, which can improve the response speed of photographing and improve the photographing experience.

[0048] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0050] Figure 1 is an image processing method flowchart shown by an embodiment of the present disclosure Figure One .

[0051] Figure 2 is an image processing method flowchart in related art shown by an embodiment of the present disclosure

[0052] Figure 3 is an image processing method flowchart shown by an embodiment of the present disclosure Figure Two .

[0053] Figure 4 is an image processing method flowchart shown by an embodiment of the present disclosure Figure Three .

[0054] Figure 5 is an image processing device diagram applied to an electronic device shown by an embodiment of the present disclosure

[0055] Figure 6 is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] The exemplary embodiments will be described in detail herein below with reference to the drawings. In the following description, the same numbers in different drawings represent the same or similar elements unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they only represent examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0057] Figure 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. Figure One As shown in Figure 1 An image processing method applied to an electronic device includes the following steps:

[0058] S101, in response to a received image acquisition request, acquiring a plurality of frames of acquisition images;

[0059] S102, performing image processing on the plurality of frames of acquisition images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail;

[0060] S103, sending the thumbnail to a camera application layer of the electronic device.

[0061] In the embodiments of the present disclosure, the electronic device includes a mobile phone, a tablet computer, a notebook computer, a palm computer, a smart watch, a sports camera, a drone, and other devices including a camera, which has a photographing function. The camera includes a camera hardware layer and a camera application layer. The camera hardware layer includes an image sensor (Image Sensor), an image processor (Image Signal Processor, ISP), and the like. The camera application layer can be an application program (Application, APP) of the camera itself in the electronic device, or can be a third-party APP capable of invoking the camera, such as a beauty camera, a filter camera, a social software capable of photographing and sharing, and the like.

[0062] In the embodiments of the present disclosure, the electronic device can respond to a received preview request for requesting to obtain preview images. The preview images are refreshed at a preset frequency of frames per second (FPS), and thus multiple preview images are obtained. In the embodiments of the present disclosure, the electronic device can send the multiple preview images to a camera application layer of the electronic device, so as to display the preview images through the camera application layer. It should be noted that the obtained preview images can be preview images processed by using an anti-shake mode for anti-shake, so as to reduce the influence of the shaking of the electronic device on the quality of the preview images. The anti-shake mode can include electronic image stabilization (EIS), optical image stabilization (OIS), hybrid image stabilization (HIS) combining the optical image stabilization and the electronic image stabilization, and the like.

[0063] In the embodiments of the present disclosure, the electronic device obtains multiple captured images in response to a received image capture request. Here, the camera application layer can obtain the multiple captured images based on the multiple preview images during the photographing process after obtaining the multiple preview images. The image capture request is a multiple photographing request, and the multiple photographing request can be a high dynamic range (HDR) request.

[0064] For example, the electronic device can obtain the multiple captured images based on a photographing mode and the multiple preview images. The photographing mode includes a zero shutter lag (ZSL) photographing mode and a non-zero shutter lag (non-ZSL) photographing mode.

[0065] For example, in the case that the photographing mode is the ZSL photographing mode and the image capture request is the HDR request, the electronic device can first store the preview images as a cache preview queue. The cache preview queue includes the generation time of each preview image and the mapping relationship of each preview image.

[0066] The obtaining of the multiple captured images based on the multiple preview images includes:

[0067] According to a time when the HDR request is received, a target preview image corresponding to the time when the HDR request is received is selected from the cache preview queue; at least two exposure values are set according to a current ambient brightness; and the target preview image is processed based on an exposure gain value and an exposure time corresponding to each exposure value to obtain a plurality of frames of acquisition images corresponding to different exposure values. Exemplarily, the at least two exposure values are three exposure values of high, medium and low, and the exposure values can be EV+9, EV0 and EV-24. In the embodiment of the present disclosure, on the one hand, if the electronic device displays the preview image through the camera application layer, the display delay can be reduced because the preview image can be quickly found from the cache preview queue, and on the other hand, the acquisition image is the image at the time when the image acquisition request is received, so that what is seen is what is obtained.

[0068] In the embodiment of the present disclosure, the electronic device performs image processing on the plurality of frames of acquisition images through the camera hardware layer of the electronic device to obtain a photograph image and a thumbnail. The image processing can include multi-frame fusion processing, image optimization processing, scaling processing and the like. Exemplarily, the electronic device can first perform fusion processing on the plurality of frames of acquisition images through the camera hardware layer of the electronic device, then perform image optimization processing on the single frame image after the fusion processing, and obtain the photograph image and the thumbnail based on the image after the image optimization processing; or the electronic device can first perform image optimization processing on the plurality of frames of acquisition images through the camera hardware layer of the electronic device, then perform fusion processing on the plurality of frames of images after the optimization processing, and obtain the photograph image and the thumbnail based on the image after the fusion processing.

[0069] In the embodiment of the present disclosure, after obtaining the photograph image and the thumbnail, the electronic device sends the thumbnail to the camera application layer of the electronic device to display the thumbnail through the camera application layer. In addition, the electronic device can send the photograph image to the camera application layer of the electronic device to display the photograph image through the camera application layer, or store the photograph image to send the stored photograph image to the camera application layer when the camera application layer needs to display the photograph image. It can be understood that the data amount of the thumbnail is smaller than that of the photograph image, and if the electronic device sends the thumbnail and the photograph image to the camera application layer at the same time, the camera application layer will first receive the thumbnail and refresh the thumbnail.

[0070] Exemplarily, the related art generally generates a thumbnail based on a photograph image after obtaining the photograph image, and then refreshes the thumbnail. Because the photograph image corresponding to the photograph frame has a relatively high resolution and needs to be processed by down-sampling before use, the image after the down-sampling processing is generally used as the thumbnail. Figure 2 is a flowchart of an image processing method in the related art shown in the embodiment of the present disclosure, as Figure 2As shown, the thumbnail is obtained by down-sampling the photographed image. Since the thumbnail needs to be generated based on the photographed image, the refresh speed of the thumbnail is slow.

[0071] In the embodiments of the present disclosure, the electronic device obtains multiple frames of captured images in response to the received image capture request, performs image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain a photographed image and a thumbnail, and sends the thumbnail to the camera application layer of the electronic device. That is, the camera hardware layer can directly perform image processing on the multiple frames of captured images, without the camera application layer calling the data of the multiple frames of captured images for image processing. In this way, the speed of image processing on the multiple frames of captured images can be improved, and the efficiency of obtaining the photographed image and the thumbnail can be improved. Moreover, the electronic device sends the thumbnail to the camera application layer to refresh the image after obtaining the thumbnail, which can improve the response speed of photographing and improve the photographing experience.

[0072] In some embodiments, the image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain a photographed image and a thumbnail includes:

[0073] performing fusion processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing;

[0074] performing image processing on the initial image to obtain the photographed image and the thumbnail.

[0075] In the embodiments of the present disclosure, the electronic device performs fusion processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing. The captured images can be images in a lossless compression format (RAW Image Format, RAW) or images in other formats. Correspondingly, the initial image after fusion processing has the same format as the captured images. The fusion processing methods include a Gaussian Pyramid-based fusion processing method, a Laplacian Pyramid-based fusion processing method, and the like. It should be noted that the initial image obtained after fusion processing can be one frame of image.

[0076] In the embodiments of the present disclosure, the electronic device performs image processing on the initial image to obtain a photographed image and a thumbnail. Specifically, the electronic device can first perform image optimization processing on the initial image, and then obtain the photographed image and the thumbnail based on the image after image optimization processing. Alternatively, the electronic device can first perform scaling processing on the initial image, and then obtain the photographed image and the thumbnail based on the image after scaling processing.

[0077] In the embodiments of the present disclosure, the electronic device fuses the multiple frames of captured images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing. Since the captured images record a lot of image information and details, the initial image with good quality can be obtained by fusing the captured images with a lot of information, so that the quality of the photographed image and the thumbnail is good. Since the photographed image and the thumbnail are both obtained based on the initial image, the field angles of view of the photographed image and the thumbnail are consistent, and the visual experience of the user is improved.

[0078] In some embodiments, the image processing on the initial image to obtain the photographed image and the thumbnail includes:

[0079] performing image optimization processing on the initial image to obtain a target image;

[0080] obtaining the photographed image and the thumbnail based on the target image.

[0081] In the embodiments of the present disclosure, the electronic device performs image optimization processing on the initial image to obtain a target image. The image optimization processing includes noise reduction, filtering, brightening, white balance, color correction, color gamut conversion processing, and the like. For example, the electronic device can first perform color gamut conversion on the initial image, and then perform noise reduction and filtering on the converted image to obtain the target image. Alternatively, the electronic device can first perform noise reduction and filtering on the initial image, and then perform color gamut conversion on the image after noise reduction and filtering, and finally perform noise reduction and filtering on the converted image to obtain the target image.

[0082] For example, the multiple frames of captured images are in RAW format, the initial image is in RAW format, and the target image is in YUV format. Figure 3 is a flowchart of an image processing method according to an embodiment of the present disclosure Figure Two As shown in Figure 3 The image processing method includes the following steps: S301, the electronic device acquires multiple frames of captured images in RAW format in response to a received image capture request; S302, the electronic device fuses the multiple frames of captured images through the camera hardware layer of the electronic device to obtain a single frame of initial image in RAW format after fusion processing; S303, the electronic device performs image optimization processing on the initial image to obtain a single frame of target image in YUV format; S304, the photographed image is obtained based on the target image; and S305, the thumbnail is obtained based on the target image.

[0083] In the embodiments of the present disclosure, the electronic device obtains the photographed image and the thumbnail based on the target image. Since the target image is obtained by performing image optimization processing on the initial image, it can be understood that the target image with good image quality can be obtained by image optimization processing, thereby improving the quality of the obtained photographed image and thumbnail. Moreover, since the photographed image and the thumbnail are both obtained based on the target image, the field of view of the photographed image and the thumbnail can be consistent on the basis of improving the quality of the thumbnail, thereby improving the visual experience of the user.

[0084] In some embodiments, the obtaining the photographed image and the thumbnail based on the target image comprises:

[0085] performing size reduction processing on the target image to obtain the thumbnail;

[0086] performing encoding processing on the target image to obtain the photographed image.

[0087] In the embodiments of the present disclosure, the electronic device performs size reduction processing on the target image to obtain the thumbnail. It can be understood that the size of the thumbnail is smaller than that of the photographed image and the target image.

[0088] In the embodiments of the present disclosure, the electronic device performs encoding processing on the target image to obtain the photographed image. It is considered that the camera application layer of the electronic device can not support the image in the format of the target image, and thus the electronic device needs to perform encoding processing on the target image. For example, if the camera application layer supports displaying the image in the Joint Photographic Experts Group (JPEG) format, and the target image or the adjusted image is in the YUV format, the YUV format image needs to be encoded to obtain the photographed image in the JPEG format.

[0089] In the embodiments of the present disclosure, the electronic device performs size reduction processing on the target image to obtain the thumbnail, and performs encoding processing on the target image to obtain the photographed image. On the one hand, it can be understood that the photographed image and the thumbnail obtained by using the size reduction processing or the encoding processing which does not affect the image content and quality based on the same data source (the target image) have consistency in content and quality. On the other hand, the electronic device performs encoding processing on the target image to obtain the photographed image in the display format adapted to the camera application layer.

[0090] In some embodiments, the performing encoding processing on the target image to obtain the photographed image comprises:

[0091] In a case where the size of the target image is inconsistent with the preset photographing size, the target image is resized, and the resized image is encoded to obtain the photographing image.

[0092] In the embodiments of the present disclosure, the preset photographing size is the size of the photographing image output by the electronic device, which can be set by the user through the camera application layer of the electronic device or can be defined by the electronic device. In the embodiments of the present disclosure, in a case where the size of the target image is inconsistent with the preset photographing size, the electronic device first resizes the target image, and then encodes the resized image to obtain the photographing image. In the embodiments of the present disclosure, in a case where the size of the target image is consistent with the preset photographing size, the electronic device does not need to resize the target image, and can directly encode the target image to obtain the photographing image.

[0093] In the embodiments of the present disclosure, in a case where the size of the target image is inconsistent with the preset photographing size, the electronic device first resizes the target image, so that the size of the resized image is adapted to the display size of the camera application layer; and then the electronic device encodes the resized image to obtain a display format adapted to the camera application layer.

[0094] In some embodiments, the image optimization processing on the initial image to obtain the target image includes:

[0095] The initial image is subjected to color gamut conversion to obtain a converted image;

[0096] The converted image is input into at least one image optimization model to obtain the target image.

[0097] If the initial image is a RAW format image, the RAW format image is displayed as gray and the color is very dull, which is completely different from what the naked eye sees when taking a photograph. It can be understood that both the photographing image and the thumbnail image are images to be displayed, and if the photographing image and the thumbnail image are RAW format images, the display image will be different from what the naked eye sees.

[0098] Therefore, in the embodiments of the present disclosure, the electronic device first converts the initial image in the color gamut to obtain a converted image. For example, if the initial image is a RAW format image, the electronic device can convert the initial image in the color gamut to obtain a converted image in YUV format.

[0099] In the embodiments of the present disclosure, the electronic device inputs the converted image into at least one image optimization model to obtain a target image. Specifically, the electronic device performs image optimization processing on the converted image in the at least one image optimization model to obtain a target image with good image quality. Exemplarily, the image optimization model includes a noise reduction model, a filtering model, a brightening model, a white balance model, a color correction model, and the like.

[0100] In the embodiments of the present disclosure, the electronic device performs color gamut conversion on the initial image to obtain a converted image, and inputs the converted image into at least one image optimization model to obtain a target image. Taking the converted image as an image in YUV format as an example, since the image in YUV format can display image information with colors, it can be understood that a photographed image and a thumbnail with colors can be obtained based on the target image in YUV format. In addition, the image in YUV format has good stability, which can improve the stability of image transmission.

[0101] In some embodiments, the method further includes:

[0102] obtaining a current ambient brightness of an environment in which the electronic device is located;

[0103] in a case where the current ambient brightness is less than a brightness threshold, obtaining a frame number of the captured image as a first frame number;

[0104] in a case where the current ambient brightness is greater than or equal to the brightness threshold, obtaining a frame number of the captured image as a second frame number;

[0105] the first frame number is greater than the second frame number.

[0106] In the embodiments of the present disclosure, the electronic device obtains a current ambient brightness of an environment in which the electronic device is located. The current ambient brightness obtained by the electronic device can be sent by a device (such as a light sensor, a brightness meter, etc.) outside the electronic device to measure the brightness, can be measured by a module (such as a light sensor module, etc.) integrated in the electronic device itself to measure the brightness, or can be obtained by the electronic device reading a preview image, and the embodiments of the present disclosure do not limit this.

[0107] In the embodiments of the present disclosure, in a case where the current environment brightness is less than the brightness threshold, the electronic device acquires a frame number of the collected images as a first frame number; in a case where the current environment brightness is greater than or equal to the brightness threshold, the electronic device acquires a frame number of the collected images as a second frame number; the first frame number is greater than the second frame number. Since in the case where the environment brightness is small, the camera may be unable to focus or may be overexposed, resulting in poor quality of the collected images, therefore, the first frame number is greater than the second frame number, and the collected images with more frame numbers are fused to obtain a clear initial image after fusion processing. In a case where the current environment in which the electronic device is located is indoor, for example, the first frame number can be 8 frames; in a case where the current environment in which the electronic device is located is outdoor, for example, the second frame number can be 3 frames.

[0108] In the embodiments of the present disclosure, the electronic device acquires the current environment brightness of the environment in which the electronic device is located; and determines the frame number of the collected images according to a comparison result of the current environment brightness and the brightness threshold. It can be seen that the method of the present disclosure can be applied to scenes with different environment brightness. In the case where the current environment brightness is small, the collected images with more frames are fused to obtain a clear initial image after fusion processing.

[0109] Figure 4 is a flowchart of an image processing method according to an embodiment of the present disclosure Figure Three As shown in Figure 4 , the method comprises the following steps:

[0110] S401, acquiring multiple frames of preview images in response to a received preview request.

[0111] In the embodiments of the present disclosure, the electronic device can acquire the preview images in response to the received preview request, which is used to request to acquire the preview images.

[0112] S402, acquiring multiple frames of collected images based on the multiple frames of preview images in response to a received image collection request.

[0113] In a case where the user clicks a photograph button or the user sends a voice instruction representing photographing, the electronic device receives the image collection request. In the embodiments of the present disclosure, in response to the received image collection request, the electronic device can select a target preview image corresponding to the time when the image collection request is received from the multiple frames of preview images based on the camera hardware layer of the electronic device, and acquire multiple frames of collected images based on the selected target preview image. Exemplarily, the multiple frames of collected images are all RAW format images.

[0114] S403, performing fusion processing on the multiple frames of collected images by the camera hardware layer of the electronic device to obtain an initial image after fusion processing.

[0115] In this embodiment of the disclosure, the electronic device performs fusion processing on multiple captured images through the camera hardware layer of the electronic device to obtain a clear fused initial image. For example, fusion processing is performed on RAW format captured images, and the resulting fused initial image is still a RAW format image.

[0116] S404. Perform color gamut conversion on the initial image to obtain the converted image.

[0117] Since RAW format images are displayed in grayscale and have very dull colors, in this embodiment of the disclosure, the electronic device performs color gamut conversion on the initial image to obtain a converted image. After color gamut conversion on the initial RAW format image, a converted YUV format image can be obtained.

[0118] S405. Input the converted image into at least one image optimization model to obtain the target image.

[0119] In this embodiment of the disclosure, the electronic device inputs the converted image into at least one image optimization model for image optimization processing to obtain a target image with good image quality.

[0120] S406. Encode the target image to obtain the photographed image.

[0121] Considering that the camera application layer of an electronic device may not support the image format for displaying the target image, in this embodiment of the disclosure, the electronic device encodes the target image to obtain a captured image, so as to obtain a captured image adapted to the display format of the camera application layer.

[0122] S407. Reduce the size of the target image to obtain a thumbnail.

[0123] Since the size of the thumbnail is smaller than that of the target image, in this embodiment of the disclosure, the electronic device reduces the size of the target image to obtain the thumbnail.

[0124] S408. Store the captured image.

[0125] In this embodiment of the disclosure, the electronic device stores the captured image and can also send the captured image to the camera application layer of the electronic device so that the captured image can be displayed through the camera application layer.

[0126] S409. Send the thumbnail to the camera application layer of the electronic device.

[0127] In this embodiment of the disclosure, the electronic device sends a thumbnail to the camera application layer of the electronic device. Since the data size of the thumbnail is smaller than that of the captured image, the camera application layer receives the thumbnail first and refreshes it.

[0128] In this embodiment, the electronic device responds to a received image acquisition request by acquiring multiple frames of images; the camera hardware layer of the electronic device performs image processing on the multiple frames of images to obtain a captured image and a thumbnail; and the thumbnail is sent to the camera application layer of the electronic device. In other words, this embodiment can directly process multiple frames of images through the camera hardware layer, without requiring the camera application layer to access the data of the multiple frames for image processing. This improves the speed of image processing for multiple frames, thereby increasing the efficiency of obtaining the captured image and thumbnail. Furthermore, after obtaining the thumbnail, this embodiment directly sends the thumbnail to the camera application layer to refresh the image, improving the shooting response speed and thus enhancing the shooting experience.

[0129] Figure 5 This is a diagram illustrating an image processing apparatus applied to an electronic device, as shown in an embodiment of this disclosure. Figure 5 As shown, the image processing apparatus 500 includes:

[0130] The first acquisition module 501 is configured to acquire multiple frames of images in response to a received image acquisition request;

[0131] The image acquisition module 502 is configured to perform image processing on the multi-frame acquired images through the camera hardware layer of the electronic device to obtain the captured image and thumbnail;

[0132] The sending module 503 is configured to send the thumbnail to the camera application layer of the electronic device.

[0133] In some embodiments, the image acquisition module 502 is further configured to perform fusion processing on the multi-frame acquired images through the camera hardware layer of the electronic device to obtain a fused initial image; and to perform image processing on the initial image to obtain the captured image and the thumbnail.

[0134] In some embodiments, the image obtaining module 502 is further configured to perform image optimization processing on the initial image to obtain a target image; and based on the target image, to obtain the captured image and the thumbnail.

[0135] In some embodiments, the image obtaining module 502 is further configured to reduce the size of the target image to obtain the thumbnail; and to encode the target image to obtain the photographed image.

[0136] In some embodiments, the image acquisition module 502 is further configured to adjust the size of the target image when the size of the target image is inconsistent with the preset photographing size, and to encode the adjusted image to obtain the photographed image.

[0137] In some embodiments, the image obtaining module 502 is further configured to perform color gamut conversion on the initial image to obtain a converted image; and input the converted image into at least one image optimization model to obtain the target image.

[0138] In some embodiments, the apparatus further includes:

[0139] The second acquisition module 504 is configured to acquire the current ambient brightness of the environment in which the electronic device is located;

[0140] The third acquisition module 505 is configured to acquire the number of frames of the captured image as a first frame number when the current ambient brightness is less than a brightness threshold; and to acquire the number of frames of the captured image as a second frame number when the current ambient brightness is greater than or equal to the brightness threshold; wherein the first frame number is greater than the second frame number.

[0141] Figure 6 This is a block diagram illustrating an electronic device according to an embodiment of this disclosure. For example, the electronic device 600 can be a mobile phone, tablet computer, laptop computer, PDA, smartwatch, action camera, drone, or other device with shooting capabilities.

[0142] Reference Figure 6 The electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0143] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0144] Memory 604 is configured to store various types of data to support the operation of electronic device 600. Examples of this data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0145] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0146] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0147] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0148] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0149] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0150] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as Wi-Fi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0151] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 820 of an electronic device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0153] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform an image processing method, the method comprising:

[0154] In response to a received image acquisition request, acquire multiple frames of images;

[0155] The camera hardware layer of the electronic device performs image processing on the multi-frame captured images to obtain photographed images and thumbnails;

[0156] The thumbnail is sent to the camera application layer of the electronic device.

[0157] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the foregoing claims.

[0158] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method is applied to an electronic device, and the method comprises: In response to a received image acquisition request, a plurality of acquisition images are acquired; An image processing is performed on the plurality of acquisition images by a camera hardware layer of the electronic device to obtain a photograph image and a thumbnail; The thumbnail is sent to a camera application layer of the electronic device.

2. The method of claim 1, wherein, The image processing on the plurality of acquisition images by the camera hardware layer of the electronic device to obtain the photograph image and the thumbnail comprises: A fusion processing is performed on the plurality of acquisition images by the camera hardware layer of the electronic device to obtain an initial image after the fusion processing; An image processing is performed on the initial image to obtain the photograph image and the thumbnail.

3. The method of claim 2, wherein, The image processing on the initial image to obtain the photograph image and the thumbnail comprises: An image optimization processing is performed on the initial image to obtain a target image; The photograph image and the thumbnail are obtained based on the target image.

4. The method of claim 3, wherein, The obtaining of the photograph image and the thumbnail based on the target image comprises: A size reduction processing is performed on the target image to obtain the thumbnail; An encoding processing is performed on the target image to obtain the photograph image.

5. The method of claim 4, wherein, The encoding processing on the target image to obtain the photograph image comprises: In a case where the size of the target image is inconsistent with a preset photograph size, a size adjustment is performed on the target image, and the photograph image is obtained by encoding the adjusted image.

6. The method of claim 3, wherein, The image optimization processing on the initial image to obtain a target image comprises: A color gamut conversion is performed on the initial image to obtain a converted image; The converted image is input into at least one image optimization model to obtain the target image.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: A current ambient brightness of an environment where the electronic device is located is acquired; In a case where the current ambient brightness is less than a brightness threshold, a frame number of the acquisition images is a first frame number; In a case where the current ambient brightness is greater than or equal to the brightness threshold, the frame number of the acquisition images is a second frame number; The first frame number is greater than the second frame number.

8. An image processing apparatus characterized by comprising: The apparatus is applied to an electronic device, and the apparatus comprises: A first acquisition module configured to acquire a plurality of acquisition images in response to a received image acquisition request; An image obtaining module configured to perform an image processing on the plurality of acquisition images by a camera hardware layer of the electronic device to obtain a photograph image and a thumbnail; A sending module configured to send the thumbnail to a camera application layer of the electronic device.

9. An electronic device, comprising: The apparatus comprises: A processor; A memory for storing processor-executable instructions; The processor is configured to perform the image processing method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the image processing method according to any one of claims 1 to 7.

Images