Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium, and the method is applied to the electronic equipment, and comprises the steps: obtaining a plurality of frames of collected images in response to a received image collection request; performing image processing on the multi-frame acquisition image through a camera hardware layer of the electronic equipment to obtain a photographed image and a thumbnail; and sending the thumbnail to a camera application layer of the electronic equipment. Through the method, the efficiency of obtaining the photographed image and the thumbnail is improved, the photographing response speed is improved, and the photographing experience is further improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method and device, an electronic device, and a storage medium. Background Art

[0002] At present, the camera speed of electronic devices such as mobile phones and tablet computers is getting faster and faster. No matter what operating system is integrated, the camera speed of electronic devices produced by various manufacturers is very fast. However, with the continuous development of camera technology, people's requirements for camera performance of electronic devices are increasing day by day.

[0003] There are many ways to improve the speed of taking photos. From the user's perspective, as long as the gallery is refreshed after taking a photo, the current photo is considered to be completed and the next photo can be taken. As for the processing of the photo image, it can be put into the post-processing queue. Therefore, many electronic device manufacturers use thumbnails to refresh the gallery. However, in related technologies, the thumbnail refresh process is relatively slow, and how to improve the speed of determining thumbnails has become a problem to be solved. Summary of the invention

[0004] The present disclosure provides an image processing method and device, an electronic device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, which is applied in an electronic device, and the method includes:

[0006] In response to the received image acquisition request, acquiring multiple frames of acquired images;

[0007] Performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails;

[0008] The thumbnail is sent to a camera application layer of the electronic device.

[0009] In some embodiments, the performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain the photographed images and thumbnails includes:

[0010] Performing fusion processing on the multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing;

[0011] The initial image is subjected to image processing to obtain the photographed image and the thumbnail.

[0012] In some embodiments, the performing image processing on the initial image to obtain the photographed image and the thumbnail includes:

[0013] Performing image optimization processing on the initial image to obtain a target image;

[0014] Based on the target image, the photographed image and the thumbnail are obtained.

[0015] In some embodiments, obtaining the photographed image and the thumbnail based on the target image includes:

[0016] Reducing the size of the target image to obtain the thumbnail;

[0017] The target image is encoded to obtain the photographed image.

[0018] In some embodiments, encoding the target image to obtain the photographed image includes:

[0019] When the size of the target image is inconsistent with the preset photographing size, the target image is resized, and the resized image is encoded to obtain the photographing image.

[0020] In some embodiments, performing image optimization processing on the initial image to obtain a target image includes:

[0021] Performing color gamut conversion on the initial image to obtain a converted image;

[0022] The converted image is input into at least one image optimization model to obtain the target image.

[0023] In some embodiments, the method further comprises:

[0024] Acquire the current ambient brightness of the environment in which the electronic device is located;

[0025] When the current environment brightness is less than a brightness threshold, obtaining a first frame number for the captured image;

[0026] When the current environment brightness is greater than or equal to the brightness threshold, the number of frames for acquiring the captured image is a second number of frames;

[0027] The first frame number is greater than the second frame number.

[0028] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing device, which is applied to an electronic device, and the device includes:

[0029] A first acquisition module is configured to acquire multiple frames of acquired images in response to a received image acquisition request;

[0030] An image acquisition module is configured to perform image processing on the multiple frames of acquired images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail;

[0031] The sending module is configured to send the thumbnail to the camera application layer of the electronic device.

[0032] In some embodiments, the image acquisition module is further configured to perform fusion processing on the multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing; and perform image processing on the initial image to obtain the photographed image and the thumbnail.

[0033] In some embodiments, the image acquisition module is further configured to perform image optimization processing on the initial image to obtain a target image; and obtain the photographed image and the thumbnail based on the target image.

[0034] In some embodiments, the image acquisition module is further configured to reduce the size of the target image to obtain the thumbnail; and encode the target image to obtain the photographed image.

[0035] In some embodiments, the image acquisition module is further configured to resize the target image when the size of the target image is inconsistent with a preset photographing size, and encode the adjusted image to obtain the photographing image.

[0036] In some embodiments, the image acquisition module is further configured to perform color gamut conversion on the initial image to obtain a converted image; and input the converted image into at least one image optimization model to obtain the target image.

[0037] In some embodiments, the apparatus further comprises:

[0038] A second acquisition module is configured to acquire the current ambient brightness of the environment in which the electronic device is located;

[0039] The third acquisition module is configured to acquire the first number of frames of the acquired image when the current ambient brightness is less than a brightness threshold; and to acquire the second number of frames of the acquired image when the current ambient brightness is greater than or equal to the brightness threshold; the first number of frames is greater than the second number of frames.

[0040] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] Wherein, the processor is configured to execute the image processing method as described in the first aspect above.

[0044] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, including:

[0045] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as described in the first aspect above.

[0046] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0047] In the embodiment of the present disclosure, the electronic device obtains multiple frames of captured images in response to a received image acquisition request; performs image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails; and sends the thumbnails to the camera application layer of the electronic device. In other words, the embodiment of the present disclosure can directly perform image processing on the multiple frames of captured images through the camera hardware layer, without the need for the camera application layer to call the data of the multiple frames of captured images and then perform image processing. In this way, the speed of image processing on the multiple frames of captured images can be improved, thereby improving the efficiency of obtaining photographic images and thumbnails. In addition, after obtaining the thumbnail, the embodiment of the present disclosure will directly send the thumbnail to the camera application layer to refresh the image, which can improve the response speed of photographing and thereby improve the photographing experience.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0050] Figure 1 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 1 .

[0051] Figure 2 It is a flow chart of an image processing method in a related technology shown in an embodiment of the present disclosure.

[0052] Figure 3 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 2 .

[0053] Figure 4 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 3 .

[0054] Figure 5 is a diagram of an image processing device applied to an electronic device shown in an embodiment of the present disclosure.

[0055] Figure 6 It is a block diagram of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] Figure 1 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 1 ,like Figure 1 As shown, the image processing method applied to the electronic device comprises the following steps:

[0058] S101, in response to a received image acquisition request, acquiring multiple frames of acquired images;

[0059] S102, performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographed images and thumbnails;

[0060] S103: Send the thumbnail to the camera application layer of the electronic device.

[0061] In the disclosed embodiments, the electronic device includes: mobile phones, tablet computers, laptop computers, PDAs, smart watches, sports cameras, drones and other devices including cameras, and the electronic device has a shooting function. Among them, the camera includes a camera hardware layer and a camera application layer; the camera hardware layer includes an image sensor (Image Sensor), an image processor (Image Signal Processor, ISP), etc.; the camera application layer can be the application (Application, APP) of the camera itself in the electronic device, or other third-party APPs that can call the camera, such as beauty cameras, filter cameras, social software that can take photos and share, etc.

[0062] In an embodiment of the present disclosure, an electronic device may respond to a received preview request, which is used to request to obtain a preview image. The preview image is refreshed at a preset frame rate per second (Frames Per Second, FPS), and it can be seen that a multi-frame preview image is obtained. In an embodiment of the present disclosure, the electronic device may send a multi-frame preview image to the camera application layer of the electronic device to display the preview image through the camera application layer. It should be noted that the obtained preview image may be a preview image after anti-shake processing using an anti-shake mode to reduce the impact of the shake of the electronic device on the quality of the preview image. The anti-shake mode may include: Electronic Image Stabilization (EIS), Optical Image Stabilization (OIS), Hybrid Image Stabilization (HIS) of a combination of optical image stabilization and electronic image stabilization, and the like.

[0063] In an embodiment of the present disclosure, the electronic device obtains multiple frames of captured images in response to a received image capture request. Here, during the photo-taking process, after obtaining multiple frames of preview images, the camera application layer of the electronic device can obtain multiple frames of captured images based on the multiple frames of preview images. The image capture request is a multiple frame photo-taking request, and the multiple frame photo-taking request can be a high dynamic range (HDR) request.

[0064] Exemplarily, the electronic device can obtain multiple frames of captured images based on the photographing mode and the multiple frames of preview images, wherein the photographing mode includes: a zero shutter lag (ZSL) photographing mode and a non-zero shutter lag (non-ZSL) photographing mode.

[0065] Taking the case where the shooting mode is the ZSL shooting mode and the image acquisition request is an HDR request as an example, the electronic device may first store the preview image as a cache preview queue; wherein the cache preview queue includes the generation time of each preview image and the mapping relationship between each preview image.

[0066] The method of acquiring multiple frames of captured images based on multiple frames of preview images includes:

[0067] According to the time when the HDR request is received, a target preview image corresponding to the time when the HDR request is received is selected from the cache preview queue; according to the current ambient brightness, at least two exposure values ​​are set; based on the exposure gain value and exposure time corresponding to each exposure value, the target preview image is processed to obtain multiple frames of captured images corresponding to different exposure values. Exemplarily, at least two exposure values ​​are high, medium, and low, and correspondingly, the exposure values ​​may be EV+9, EV0, and EV-24. In the disclosed embodiments, on the one hand, if the electronic device displays a preview image through the camera application layer, since the preview image can be quickly found from the cache preview queue, the display delay can be reduced. On the other hand, since the captured image is the image at the time when the image capture request is received, what you see is what you get.

[0068] In an embodiment of the present disclosure, the electronic device performs image processing on multiple frames of acquired images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails. The image processing may include multi-frame fusion processing, image optimization processing, scaling processing and other processing. Exemplarily, the electronic device may first perform fusion processing on multiple frames of acquired images through the camera hardware layer of the electronic device, then perform image optimization processing on the single frame image after the fusion processing, and obtain photographic images and thumbnails based on the image after the image optimization processing; it may also first perform image optimization processing on multiple frames of acquired images through the camera hardware layer of the electronic device, then perform fusion processing on the multiple frames of images after the optimization processing, and obtain photographic images and thumbnails based on the image after the fusion processing.

[0069] In the embodiment of the present disclosure, after obtaining the photographic image and the thumbnail, the electronic device sends the thumbnail to the camera application layer of the electronic device so that the thumbnail is displayed through the camera application layer. In addition, the electronic device can send the photographic image to the camera application layer of the electronic device so that the photographic image can be displayed through the camera application layer; the photographic image can also be stored so that when the camera application layer needs to display the photographic image, the stored photographic image can be sent to the camera application layer. It can be understood that the data volume of the thumbnail is smaller than the data volume of the photographic image. If the electronic device sends the thumbnail and the photographic image to the camera application layer at the same time, the camera application layer will receive the thumbnail first and refresh the thumbnail.

[0070] For example, related technologies usually generate thumbnails based on the captured images after acquiring the captured images, and then refresh them. Because the resolution of the captured images corresponding to the photo frames is relatively high, they need to be downsampled before they can be used. Therefore, the downsampled images are usually used as thumbnails. Figure 2 is a flow chart of an image processing method in a related technology shown in an embodiment of the present disclosure, such as Figure 2As shown, the captured image is downsampled to obtain a thumbnail. Since the acquisition of the thumbnail depends on the captured image generated in advance, there is a problem that the thumbnail refresh speed is slow.

[0071] In this regard, in an embodiment of the present disclosure, the electronic device obtains multiple frames of captured images in response to a received image capture request; performs image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails; and sends the thumbnails to the camera application layer of the electronic device. In other words, the embodiment of the present disclosure can directly perform image processing on the multiple frames of captured images through the camera hardware layer, without the need for the camera application layer to call the data of the multiple frames of captured images and then perform image processing. In this way, the speed of image processing on the multiple frames of captured images can be improved, thereby improving the efficiency of obtaining photographic images and thumbnails. In addition, after obtaining the thumbnail, the embodiment of the present disclosure will directly send the thumbnail to the camera application layer to refresh the image, which can improve the response speed of photographing and thereby improve the photographing experience.

[0072] In some embodiments, the performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain the photographed images and thumbnails includes:

[0073] Performing fusion processing on the multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing;

[0074] The initial image is subjected to image processing to obtain the photographed image and the thumbnail.

[0075] In an embodiment of the present disclosure, the electronic device performs fusion processing on multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing. The acquired image may be an image in a lossless compression format (RAW Image Format, RAW) or an image in another format; correspondingly, the format of the initial image after fusion processing is the same as that of the acquired image. The fusion processing method includes: a fusion processing method based on a Gaussian Pyramid, a fusion processing method based on a Laplacian Pyramid, etc. It should be noted that the initial image obtained after the fusion processing may be a frame of image.

[0076] In the disclosed embodiment, the electronic device performs image processing on the initial image to obtain a photographic image and a thumbnail. Specifically, the electronic device may first perform image optimization processing on the initial image, and then obtain the photographic image and the thumbnail based on the image after the image optimization processing; the electronic device may also first perform scaling processing on the initial image, and then obtain the photographic image and the thumbnail based on the image after the scaling processing.

[0077] In the disclosed embodiment, the electronic device performs fusion processing on multiple frames of captured images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing. Since the captured images record a lot of image information and details, the fusion processing of the captured images with a lot of information can obtain a high-quality initial image, thereby obtaining a high-quality photographic image and thumbnail quality; and since the photographic image and thumbnail are both obtained based on the initial image, the field of view of the photographic image and the thumbnail are consistent, which improves the user's visual experience.

[0078] In some embodiments, the performing image processing on the initial image to obtain the photographed image and the thumbnail includes:

[0079] Performing image optimization processing on the initial image to obtain a target image;

[0080] Based on the target image, the photographed image and the thumbnail are obtained.

[0081] In an embodiment of the present disclosure, the electronic device performs image optimization processing on the initial image to obtain a target image. The image optimization processing includes: noise reduction, filtering, brightening, white balance, color correction, color gamut conversion processing and other processing. Exemplarily, the electronic device may first perform color gamut conversion on the initial image, and perform noise reduction, filtering and other processing on the converted image to obtain the target image; the electronic device may also first perform noise reduction, filtering and other processing on the initial image, and then perform color gamut conversion on the image after noise reduction, filtering and other processing, and finally perform noise reduction, filtering and other processing on the converted image to obtain the target image.

[0082] Exemplarily, the multiple frames of acquired images are in RAW format, the initial image is in RAW format, and the target image is in YUV format. Figure 3 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 2 ,like Figure 3 As shown, the image processing method includes the following steps: S301, the electronic device obtains a multi-frame captured image in RAW format in response to a received image acquisition request; S302, the multi-frame captured image is fused through the camera hardware layer of the electronic device to obtain a single-frame initial image in RAW format after the fusion processing; S303, the electronic device performs image optimization processing on the initial image to obtain a single-frame target image in YUV format; S304, based on the target image, a photographed image is obtained; S305, based on the target image, a thumbnail is obtained.

[0083] In the disclosed embodiment, the electronic device obtains a photographic image and a thumbnail based on the target image. Since the target image is obtained after image optimization processing based on the initial image, it can be understood that a target image with good image quality can be obtained through image optimization processing, thereby improving the quality of the obtained photographic image and thumbnail; and since both the photographic image and the thumbnail are obtained based on the target image, on the basis of improving the thumbnail quality, the field of view of the photographic image and the thumbnail can be made consistent, thereby improving the user's visual experience.

[0084] In some embodiments, obtaining the photographed image and the thumbnail based on the target image includes:

[0085] Reducing the size of the target image to obtain the thumbnail;

[0086] The target image is encoded to obtain the photographed image.

[0087] In the embodiment of the present disclosure, the electronic device reduces the size of the target image to obtain a thumbnail. It can be understood that the size of the thumbnail is smaller than the photographed image and also smaller than the target image.

[0088] In the disclosed embodiment, the electronic device encodes the target image to obtain a photographed image. Considering that the camera application layer of the electronic device may not support the display of the image in the format of the target image, the electronic device needs to encode the target image. Exemplarily, if the camera application layer supports the display of images in the format of the Joint Photographic Experts Group (JPEG) still image data compression standard, and the target image or the adjusted image is an image in the YUV format, the image in the YUV format needs to be encoded to obtain a photographed image in the JPEG format.

[0089] In the disclosed embodiment, the electronic device reduces the size of the target image to obtain a thumbnail; and encodes the target image to obtain a photographic image. On the one hand, it can be understood that based on the same data source (target image), the content and quality of the photographic image and the thumbnail obtained by using a reduction process or an encoding process that does not affect the image content and quality are consistent; on the other hand, the electronic device encodes the target image to obtain a photographic image in a display format that is adapted to the camera application layer.

[0090] In some embodiments, encoding the target image to obtain the photographed image includes:

[0091] When the size of the target image is inconsistent with the preset photographing size, the target image is resized, and the resized image is encoded to obtain the photographing image.

[0092] In the disclosed embodiment, the preset photographing size is the size of the photographing image output by the electronic device, which may be set by the user through the camera application layer of the electronic device, or may be customized by the electronic device. In the disclosed embodiment, when the size of the target image is inconsistent with the preset photographing size, the electronic device first resizes the target image, and then encodes the adjusted image to obtain the photographing image. In the disclosed embodiment, when the size of the target image is consistent with the preset photographing size, the electronic device does not need to resize the target image, and may directly encode the target image to obtain the photographing image.

[0093] In the disclosed embodiment, when the size of the target image is inconsistent with the preset photo taking size, the electronic device first resizes the target image so that the size of the adjusted image is adapted to the display size of the camera application layer; the electronic device then encodes the adjusted image to obtain a display format adapted to the camera application layer.

[0094] In some embodiments, performing image optimization processing on the initial image to obtain a target image includes:

[0095] Performing color gamut conversion on the initial image to obtain a converted image;

[0096] The converted image is input into at least one image optimization model to obtain the target image.

[0097] If the original image is in RAW format, and the RAW image is displayed in gray and very dim, the display effect is completely different from what the naked eye sees when taking the photo. It is understandable that both the photographed image and the thumbnail are images to be displayed. If the photographed image and the thumbnail are in RAW format, the displayed image will be different from what the naked eye sees.

[0098] Therefore, in the embodiment of the present disclosure, the electronic device first performs color gamut conversion on the initial image to obtain a converted image. Exemplarily, the initial image is an image in RAW format, and the electronic device can perform color gamut space conversion on the initial image to obtain a converted image in YUV format.

[0099] In the disclosed embodiment, the electronic device then inputs the converted image into at least one image optimization model to obtain a target image. Specifically, the electronic device performs image optimization processing on the converted image in the at least one image optimization model to obtain a target image with good image quality. Exemplarily, the image optimization model includes a noise reduction model, a filtering model, a brightening model, a white balance model, a color correction model, etc.

[0100] In the disclosed embodiment, the electronic device performs color gamut conversion on the initial image to obtain a converted image; the converted image is input into at least one image optimization model to obtain a target image. Taking the converted image as an image in YUV format as an example, since the image in YUV format can display image information in color, it can be understood that based on the target image in YUV format, a colored photographic image and thumbnail can be obtained; in addition, the image in YUV format has good stability, which can improve the stability of image transmission.

[0101] In some embodiments, the method further comprises:

[0102] Acquire the current ambient brightness of the environment in which the electronic device is located;

[0103] When the current environment brightness is less than a brightness threshold, obtaining a first frame number for the captured image;

[0104] When the current environment brightness is greater than or equal to the brightness threshold, the number of frames for acquiring the captured image is a second number of frames;

[0105] The first frame number is greater than the second frame number.

[0106] In an embodiment of the present disclosure, the electronic device obtains the current ambient brightness of the environment in which the electronic device is located. The electronic device may obtain the current ambient brightness by sending it through a device for measuring brightness other than the electronic device (such as a light sensor, a brightness meter, etc.), or by measuring it through a module for measuring brightness integrated in the electronic device itself (such as a light sensor module, etc.), or by reading a preview image from the electronic device, which is not limited in the embodiment of the present disclosure.

[0107] In the embodiment of the present disclosure, when the current environment brightness is less than the brightness threshold, the number of frames of the electronic device to obtain the captured image is the first number of frames; when the current environment brightness is greater than or equal to the brightness threshold, the number of frames of the electronic device to obtain the captured image is the second number of frames; the first number of frames is greater than the second number of frames. Since the camera may be unable to focus or overexposed when the environment brightness is low, resulting in poor quality of the captured image, the first number of frames is greater than the second number of frames, and a larger number of frames of captured images are used for fusion processing to obtain a clear initial image after fusion processing. Among them, when the current environment brightness is less than the brightness threshold, taking the current electronic device as an indoor environment as an example, the first number of frames can be 8 frames; when the current environment brightness is greater than or equal to the brightness threshold, taking the current electronic device as an outdoor environment as an example, the second number of frames can be 3 frames.

[0108] In the embodiment of the present disclosure, the electronic device obtains the current ambient brightness of the environment in which the electronic device is located; and determines the number of frames of the captured image based on the comparison result of the current ambient brightness and the brightness threshold. It can be seen that the method of the present disclosure can be applied to scenes with different ambient brightness. In scenes with low current ambient brightness, a fusion process is performed by fusion processing of the captured images of more frames to obtain a clear initial image after fusion processing.

[0109] Figure 4 This is a schematic diagram of an image processing method shown in an embodiment of the present disclosure. Figure 3 ,like Figure 4 As shown, the following steps are included:

[0110] S401: Acquire multiple frames of preview images in response to a received preview request.

[0111] In an embodiment of the present disclosure, the electronic device may obtain the preview image in response to a received preview request, where the preview request is used to request obtaining the preview image.

[0112] S402: In response to the received image acquisition request, acquire multiple frames of acquired images based on multiple frames of preview images.

[0113] When a user clicks a photo button or sends a voice command to take a photo, the electronic device receives an image acquisition request. In an embodiment of the present disclosure, the electronic device, in response to the received image acquisition request, may select a target preview image corresponding to the time when the image acquisition request is received from multiple preview images based on the camera hardware layer of the electronic device, and acquire multiple acquired images based on the selected target preview image. Exemplarily, the multiple acquired images are all in RAW format.

[0114] S403: performing fusion processing on multiple frames of captured images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing.

[0115] In the disclosed embodiment, the electronic device performs fusion processing on multiple frames of captured images through the camera hardware layer of the electronic device to obtain a clear fused initial image. Exemplarily, the fusion processing is performed on the RAW format captured images, and the obtained fused initial image is still an image in the RAW format.

[0116] S404: Perform color gamut conversion on the initial image to obtain a converted image.

[0117] Since the image in RAW format is displayed as gray and the color is very dim, in this regard, in the embodiment of the present disclosure, the electronic device performs color gamut conversion on the initial image to obtain a converted image. After the color gamut conversion is performed on the initial image in RAW format, a converted image in YUV format can be obtained.

[0118] S405: Input the converted image into at least one image optimization model to obtain a target image.

[0119] In the disclosed embodiment, the electronic device inputs the converted image into at least one image optimization model for image optimization processing to obtain a target image with good image quality.

[0120] S406: Encode the target image to obtain the photographed image.

[0121] Considering that the camera application layer of the electronic device may not support images in the format of displaying the target image, in an embodiment of the present disclosure, the electronic device encodes the target image to obtain a photographic image to obtain a photographic image in a display format suitable for the camera application layer.

[0122] S407: Reduce the size of the target image to obtain a thumbnail.

[0123] Since the size of the thumbnail is smaller than that of the target image, in the embodiment of the present disclosure, the electronic device reduces the size of the target image to obtain the thumbnail.

[0124] S408: Store the photographed image.

[0125] In the embodiment of the present disclosure, the electronic device stores the photographed image and may also send the photographed image to the camera application layer of the electronic device so that the photographed image can be displayed through the camera application layer.

[0126] S409: Send the thumbnail to the camera application layer of the electronic device.

[0127] In the disclosed embodiment, the electronic device sends the thumbnail to the camera application layer of the electronic device. Since the data volume of the thumbnail is smaller than the data volume of the photographed image, the camera application layer will first receive the thumbnail and refresh the thumbnail.

[0128] In the embodiment of the present disclosure, the electronic device obtains multiple frames of captured images in response to a received image acquisition request; performs image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails; and sends the thumbnails to the camera application layer of the electronic device. In other words, the embodiment of the present disclosure can directly perform image processing on the multiple frames of captured images through the camera hardware layer, without the need for the camera application layer to call the data of the multiple frames of captured images and then perform image processing. In this way, the speed of image processing on the multiple frames of captured images can be improved, thereby improving the efficiency of obtaining photographic images and thumbnails. In addition, after obtaining the thumbnail, the embodiment of the present disclosure will directly send the thumbnail to the camera application layer to refresh the image, which can improve the response speed of photographing and thereby improve the photographing experience.

[0129] Figure 5 FIG. 1 is a diagram of an image processing device applied to an electronic device according to an embodiment of the present disclosure. Figure 5 As shown, the image processing device 500 includes:

[0130] The first acquisition module 501 is configured to acquire multiple frames of acquired images in response to a received image acquisition request;

[0131] An image acquisition module 502 is configured to perform image processing on the multiple frames of acquired images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail;

[0132] The sending module 503 is configured to send the thumbnail to the camera application layer of the electronic device.

[0133] In some embodiments, the image acquisition module 502 is further configured to perform fusion processing on the multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing; and perform image processing on the initial image to obtain the photographed image and the thumbnail.

[0134] In some embodiments, the image obtaining module 502 is further configured to perform image optimization processing on the initial image to obtain a target image; and obtain the photographed image and the thumbnail based on the target image.

[0135] In some embodiments, the image acquisition module 502 is further configured to reduce the size of the target image to obtain the thumbnail; and encode the target image to obtain the photographed image.

[0136] In some embodiments, the image obtaining module 502 is further configured to adjust the size of the target image when the size of the target image is inconsistent with a preset photographing size, and encode the adjusted image to obtain the photographing image.

[0137] In some embodiments, the image acquisition module 502 is further configured to perform color gamut conversion on the initial image to obtain a converted image; and input the converted image into at least one image optimization model to obtain the target image.

[0138] In some embodiments, the apparatus further comprises:

[0139] A second acquisition module 504 is configured to acquire the current ambient brightness of the environment in which the electronic device is located;

[0140] The third acquisition module 505 is configured to acquire the first number of frames of the acquired image when the current ambient brightness is less than a brightness threshold; and acquire the second number of frames of the acquired image when the current ambient brightness is greater than or equal to the brightness threshold; the first number of frames is greater than the second number of frames.

[0141] Figure 6 600 is a block diagram of an electronic device shown in an embodiment of the present disclosure. For example, the electronic device 600 may be a device with a shooting function, such as a mobile phone, a tablet computer, a laptop computer, a PDA, a smart watch, a sports camera, and a drone.

[0142] Reference Figure 6 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0143] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0144] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0145] The power component 606 provides power to the various components of the electronic device 600. The power component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0146] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0147] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0148] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0149] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0150] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0151] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by the processor 820 of the electronic device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform an image processing method, the method comprising:

[0154] In response to the received image acquisition request, acquiring multiple frames of acquired images;

[0155] Performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails;

[0156] The thumbnail is sent to a camera application layer of the electronic device.

[0157] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims above.

[0158] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: Applied to electronic equipment, the method comprises: In response to the received image acquisition request, acquiring multiple frames of acquired images; Performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographic images and thumbnails; The thumbnail is sent to a camera application layer of the electronic device.

2. The method according to claim 1, characterized in that The step of performing image processing on the multiple frames of captured images through the camera hardware layer of the electronic device to obtain photographed images and thumbnails includes: Performing fusion processing on the multiple frames of acquired images through the camera hardware layer of the electronic device to obtain an initial image after fusion processing; The initial image is subjected to image processing to obtain the photographed image and the thumbnail.

3. The method according to claim 2, characterized in that The performing image processing on the initial image to obtain the photographed image and the thumbnail includes: Performing image optimization processing on the initial image to obtain a target image; Based on the target image, the photographed image and the thumbnail are obtained.

4. The method according to claim 3, characterized in that The obtaining the photographed image and the thumbnail based on the target image includes: Reducing the size of the target image to obtain the thumbnail; The target image is encoded to obtain the photographed image.

5. The method according to claim 4, characterized in that The encoding process of the target image to obtain the photographed image includes: When the size of the target image is inconsistent with the preset photographing size, the target image is resized, and the resized image is encoded to obtain the photographing image.

6. The method according to claim 3, characterized in that The performing image optimization processing on the initial image to obtain a target image includes: Performing color gamut conversion on the initial image to obtain a converted image; The converted image is input into at least one image optimization model to obtain the target image.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire the current ambient brightness of the environment in which the electronic device is located; When the current environment brightness is less than a brightness threshold, obtaining a first frame number for the captured image; When the current environment brightness is greater than or equal to the brightness threshold, the number of frames for acquiring the captured image is a second number of frames; The first frame number is greater than the second frame number.

8. An image processing device, characterized in that: Applied to electronic equipment, the device comprises: A first acquisition module is configured to acquire multiple frames of acquired images in response to a received image acquisition request; An image acquisition module is configured to perform image processing on the multiple frames of acquired images through a camera hardware layer of the electronic device to obtain a photographed image and a thumbnail; The sending module is configured to send the thumbnail to the camera application layer of the electronic device.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the image processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method as claimed in any one of claims 1 to 7.

Images