Image shooting method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an image shooting method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: in response to a received first request, acquiring a first image and a second image by a hardware abstraction layer of the electronic equipment; wherein the first request is used for requesting the camera to execute shooting operation; the first image is an original image shot based on a camera; the second image is a preview thumbnail of the first image; the hardware abstraction layer sends the second image to the application framework layer and performs image post-processing on the first image; under the condition that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image; wherein the resolution of the third image is higher than that of the second image; in response to the received second request, the hardware abstraction layer sends a third image to the application framework layer; the second request is used for requesting to acquire the first image after image post-processing.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of photographing, and in particular to an image photographing method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] With the wide use of electronic equipment, photographing and video recording using electronic equipment has become a daily behavior in people's life.

[0003] In order to improve the user experience and make the user feel that the photographing speed of the electronic equipment is improved, the preview thumbnail image is usually returned when the user photographs, so that the user regards the time when the electronic equipment returns the preview thumbnail image as the time when the current photographing is completed, and the actually photographed image is returned after a series of image processing is completed, so that the user can view a clearer photographed image. However, this photographing method is prone to cause the loss of the photographed image, which seriously affects the user experience. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides an image photographing method and device, electronic equipment, storage medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, an image photographing method is provided, comprising:

[0006] In response to a received first request, a hardware abstraction layer of an electronic equipment acquires a first image and a second image; wherein the first request is used to request a camera to perform a photographing operation; the first image is an original image obtained based on the camera; and the second image is a preview thumbnail image of the first image.

[0007] The hardware abstraction layer sends the second image to an application framework layer, and performs image post-processing on the first image.

[0008] In the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image; wherein the resolution of the third image is higher than that of the second image.

[0009] In response to a received second request, the hardware abstraction layer sends the third image to the application framework layer; and the second request is used to request to acquire the first image on which the image post-processing is completed.

[0010] In some embodiments, the hardware abstraction layer sends the second image to the application framework layer, and performs image post-processing on the first image, comprising:

[0011] The hardware abstraction layer performs the image post-processing of the first image through a first thread;

[0012] The second image obtained is sent to the application framework layer through a second thread; wherein the second thread is a thread executed concurrently with the first thread.

[0013] In some embodiments, the hardware abstraction layer comprises a first abstraction layer and a second abstraction layer;

[0014] The method further comprises:

[0015] The first abstraction layer generates a post-processing task based on the obtained first image, and adds the post-processing task to a task queue;

[0016] The second abstraction layer executes at least one of the post-processing tasks in the task queue in sequence;

[0017] In the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image, comprising:

[0018] In the case that the post-processing task for the first image fails, the second abstraction layer deletes the post-processing task for the first image from the task queue;

[0019] The second abstraction layer obtains the second image associated with the first image, and performs image super-resolution processing on the second image to obtain the third image.

[0020] In some embodiments, the method further comprises:

[0021] The first abstraction layer writes the obtained at least one frame of preview image into a preview cache queue;

[0022] The second abstraction layer determines the second image from the preview cache queue according to a first request, and stores the second image to a standby cache area; wherein the images stored in the standby cache area are used for recovery of the shooting result in the case that the shooting result of the electronic device is lost.

[0023] In some embodiments, in response to the received image shooting request, the hardware abstraction layer of the terminal obtains a first image and a second image, comprising:

[0024] In response to the image shooting request, the first abstraction layer obtains scene parameters of a current shooting scene;

[0025] The second abstraction layer determines a shooting strategy corresponding to the scene parameters based on the scene parameters;

[0026] The second abstraction layer sends the image capturing request carrying the shooting strategy to the first abstraction layer.

[0027] The first abstraction layer controls the camera to capture the first image and the second image according to the shooting strategy.

[0028] In some embodiments, the method further comprises:

[0029] The application framework layer applies a pre-storage area; the pre-storage area is used to store a shooting image captured when the camera performs the shooting operation.

[0030] The application framework layer sends the second request to the hardware abstraction layer, wherein the second request carries indication information indicating a storage address of the pre-storage area.

[0031] The hardware abstraction layer sends the third image to the application framework layer in response to the received second request, including:

[0032] The hardware abstraction layer stores the third image into the pre-storage area in response to the second request.

[0033] According to a second aspect of the embodiments of the present disclosure, an image capturing device is provided, including:

[0034] An acquisition module is configured to acquire, in response to a received first request, a first image and a second image by a hardware abstraction layer of an electronic device; wherein the first request is used to request a camera to perform a shooting operation; the first image is an original image captured by the camera; and the second image is a preview thumbnail of the first image.

[0035] A first processing module is configured to send, by the hardware abstraction layer, the second image to an application framework layer, and perform image post-processing on the first image.

[0036] A second processing module is configured to, in a case where the image post-processing on the first image fails, perform image super-resolution processing on the second image to obtain a third image; wherein a resolution of the third image is higher than a resolution of the second image.

[0037] A sending module is configured to, in response to a received second request, send, by the hardware abstraction layer, the third image to the application framework layer; the second request is used to request to acquire the first image on which the image post-processing is completed.

[0038] In some embodiments, the first processing module is configured to:

[0039] The hardware abstraction layer performs the image post-processing on the first image through a first thread;

[0040] The second image obtained is sent to the application framework layer through a second thread; wherein the second thread is a thread executed concurrently with the first thread.

[0041] In some embodiments, the hardware abstraction layer comprises a first abstraction layer and a second abstraction layer;

[0042] The first processing module is configured to generate a post-processing task based on the obtained first image by the first abstraction layer, and add the post-processing task into a task queue; the second abstraction layer executes at least one post-processing task in the task queue in sequence;

[0043] The second processing module is configured to, in the case that the post-processing task for the first image fails to be executed, delete the post-processing task for the first image from the task queue by the second abstraction layer; the second abstraction layer obtains the second image associated with the first image, and performs image super-resolution processing on the second image to obtain the third image.

[0044] In some embodiments, the first processing module is configured to:

[0045] The first abstraction layer writes the obtained at least one frame of preview image into a preview cache queue;

[0046] The second abstraction layer determines the second image from the preview cache queue according to a first request, and stores the second image into a standby cache area; wherein the images stored in the standby cache area are used for recovery of a shooting result in the case that the shooting result of the electronic device is lost.

[0047] In some embodiments, the obtaining module is configured to:

[0048] In response to the image shooting request, the first abstraction layer obtains scene parameters of a current shooting scene;

[0049] The second abstraction layer determines a shooting strategy corresponding to the scene parameters based on the scene parameters;

[0050] The second abstraction layer sends the image shooting request carrying the shooting strategy to the first abstraction layer;

[0051] The first abstraction layer controls the camera to collect the first image and the second image according to the shooting strategy.

[0052] In some embodiments, the sending module is configured to apply for a pre-storage area by the application framework layer; the pre-storage area is used to store a shooting image collected when the camera performs the shooting operation; and the application framework layer sends the second request to the hardware abstraction layer, wherein the second request carries indication information indicating a storage address of the pre-storage area.

[0053] The sending module is further configured to store the third image into the pre-storage area in response to the second request.

[0054] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0055] a processor;

[0056] a memory for storing computer programs or instructions;

[0057] The processor executes the computer programs or instructions to implement the steps of the method according to the first aspect of the embodiments of the present disclosure.

[0058] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, which stores computer programs or instructions, and when the computer programs or instructions in the storage medium are executed by a processor, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.

[0059] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, and when the computer programs or instructions are executed by a processor, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.

[0060] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0061] After the hardware abstraction layer of the electronic device in the embodiments of the present disclosure receives a first request for requesting a camera to perform a shooting operation, a first image and a second image associated with the shooting operation are obtained; the first image is an original image shot by the camera when performing the shooting operation; and the second image is a preview thumbnail image consistent with the image content of the first image. The hardware abstraction layer can send the second image to the application framework layer and perform image post-processing on the first image. Thus, the application framework layer can return the second image to the image shooting application in advance during the image post-processing of the first image, so that the user feels that the shooting speed of the electronic device is improved.

[0062] In addition, in a case where the image post-processing of the first image fails, the hardware abstraction layer obtains a second image associated with the first image, performs image super-resolution processing on the second image, and returns a third image obtained by the image super-resolution processing to the application framework layer to replace the first image, thereby reducing the loss of a shooting result caused by the failure of the post-processing task of the first image, and improving the user experience.

[0063] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure, together with the description.

[0065] Figure 1 is a schematic diagram of an image shooting method according to the related art.

[0066] Figure 2 is a flowchart of an image shooting method according to an exemplary embodiment.

[0067] Figure 3 is a schematic diagram of a shooting process of an electronic device according to an exemplary embodiment Figure 1 .

[0068] Figure 4 is a schematic diagram of a shooting process of an electronic device according to an exemplary embodiment Figure 2 .

[0069] Figure 5 is a block diagram of an image shooting device according to an exemplary embodiment.

[0070] Figure 6 is a structural block diagram of an electronic device 600 according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] The exemplary embodiments will be described in detail hereinafter with reference to the attached drawings. In the following description, the same numbers are used to indicate the same elements throughout the drawings, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not representative of all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0072] As Figure 1 shown, Figure 1is a schematic diagram of an image shooting method according to the related art. During shooting, the hardware abstraction layer returns a preview thumbnail (i.e., a small image) returned by the hardware driver layer to the application layer first, so as to display the preview thumbnail on the shooting interface of the electronic device, and make the user feel that the current shooting has been completed. After returning the preview thumbnail, the hardware abstraction layer performs image post-processing on the shooting image (i.e., a large image) returned by the hardware driver layer, and returns the shooting image after image post-processing to the application layer, so as to display a shooting image with better clarity and quality.

[0073] However, due to various uncertain factors during shooting, such as image processing algorithm processing timeout, image acquisition cache timeout, etc., the image processing on the actually shot image fails, the shooting image cannot be returned normally, the shooting image of the current shooting is lost, and the user experience is affected.

[0074] Based on this, the image shooting method provided in the embodiments of the present disclosure is shown in Figure 2 Figure 2 is a flowchart of an image shooting method according to an example embodiment. The method mainly includes the following steps:

[0075] Step S201, in response to a received first request, the hardware abstraction layer of the electronic device acquires a first image and a second image; wherein the first request is used to request the camera to perform a shooting operation; the first image is an original image obtained based on the camera shooting; and the second image is a preview thumbnail of the first image;

[0076] Step S202, the hardware abstraction layer sends the second image to the application framework layer, and performs image post-processing on the first image;

[0077] Step S203, in the case that the image post-processing on the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image; wherein the resolution of the third image is higher than that of the second image;

[0078] Step S204, in response to a received second request, the hardware abstraction layer sends the third image to the application framework layer; the second request is used to request to acquire the first image after image post-processing.

[0079] The image shooting method shown in the embodiments of the present disclosure can be applied to an electronic device configured with a camera, and the electronic device can include a mobile terminal and a fixed terminal. The mobile terminal can be a smartphone, a tablet computer, a notebook computer, or a wearable electronic device, etc. The fixed terminal can be a desktop computer, a smart large screen, etc.

[0080] ​It should be noted that the framework of the operating system (such as the Android system) of the electronic device can include an application layer, an application framework layer, a hardware abstraction layer, and a hardware driver layer.

[0081] The application layer is composed of all applications running in the electronic device, including pre-installed system applications and user-installed third-party applications. The application layer manages all applications in the electronic device.

[0082] The application framework layer is the core of application development and provides a basic Application Programming Interface (API) framework for developers to develop applications.

[0083] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware.

[0084] The hardware driver layer is used to provide an operating interface for the hardware device for the upper layer program, and to implement the driver program of the device. So that the upper layer program can not care about the internal implementation of the operating hardware device, but only needs to call the interface of the driver.

[0085] It should be noted that the layers are usually connected through binder, callback function or system call; the hardware driver layer is connected with the hardware device through I / O control command.

[0086] In step S201, the hardware abstraction layer acquires the first image and the second image in response to the image capturing request after receiving the first request sent by the application framework layer.

[0087] Here, the first image is an original image captured by the camera of the electronic device; the second image is a preview thumbnail of the first image. It can be understood that the image content of the first image and the image content of the second image are the same, but the image size of the first image is larger than the image size of the second image.

[0088] The first request is used to request the camera to perform a shooting operation; it should be noted that the user can initiate a shooting instruction through an image capturing application, and the application framework layer can send the first request to the hardware abstraction layer after receiving the shooting instruction, so that the hardware abstraction layer calls the camera through the hardware driver layer to perform the shooting operation.

[0089] It should be noted that in order to improve the image quality collected by the electronic device, the collected first image usually needs to be image-processed, and the first image after image processing is returned to the image collection application to complete the image collection operation. As for the second image, since the second image is a preview image displayed on the preview interface of the electronic device during image collection, the second image does not need to be image-processed.

[0090] In step S202, after obtaining the first image and the second image, the hardware abstraction layer first sends the second image to the application framework layer, so that the application framework layer returns the second image to the image shooting application, to display the second image on the shooting interface of the electronic device; at the same time, the hardware abstraction layer continues to perform image post-processing on the obtained first image.

[0091] Here, the image post-processing can include, but is not limited to, at least one of beautifying processing, blurring processing, filter processing, high dynamic range imaging, and multi-frame processing.

[0092] It can be understood that, as users have higher and higher requirements for the image quality of the first image obtained by the electronic device, in order to improve the image quality of the first image, the calculation amount of image post-processing on the first image collected by the camera is increasing, resulting in a longer post-processing time.

[0093] In order to improve the shooting speed of the electronic device, the hardware abstraction layer can first send the second image that does not need image post-processing to the application framework layer, so that the application framework layer returns the second image to the image shooting application; so that the user feels that this time of shooting has been completed. For the first image that needs further processing, the hardware abstraction layer continues to perform image post-processing on the first image to obtain a processed first image.

[0094] In other words, when the application framework layer returns the second image to the image shooting application, the electronic device has not completed the shooting task this time, and the first image still needs to be processed. But since the image shooting application will display the second image on the preview interface after receiving the second image, it will make the user feel that the electronic device has completed this time of shooting task, so as to provide the user with a "one-click shooting" shooting experience.

[0095] In step S203, in the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image.

[0096] Here, the resolution of the third image is higher than that of the second image.

[0097] It should be noted that image super-resolution processing, i.e., image super-resolution (SR) processing, refers to a kind of image processing calculation that processes a low-resolution image to restore a high-resolution image. For example, an image super-resolution model can be used to perform image super-resolution processing on the second image to obtain a third image.

[0098] It can be understood that, in the process of image post-processing of the first image, the image post-processing of the first image can fail due to processing timeout of the post-processing algorithm, timeout of obtaining a buffer, and the like, so that the processed first image cannot be obtained, the processed first image cannot be returned to the image shooting application, the shooting result is lost, and the user experience is affected.

[0099] In this case, the hardware abstraction layer can utilize the second image with the same image content as the first image, perform image super-resolution processing on the second image, reconstruct a third image with higher resolution, return the third image as the shooting result to the image shooting application, and reduce the occurrence of the shooting result loss.

[0100] In step S204, after the hardware abstraction layer receives the second request sent by the application framework layer, the hardware abstraction layer can send the third image to the application framework layer in response to the second request, so that the application framework layer returns the third image to the image shooting application.

[0101] Here, the second request is used to request to obtain the first image after the image post-processing is completed. It can be understood that the second request can be initiated by the image shooting application after receiving the second image; or the second request can be initiated by the application framework layer in response to the user initiating the first image storage instruction or the first image viewing instruction through the image shooting application.

[0102] It is worth noting that the receiving time of the second request received by the hardware abstraction layer can be after the hardware abstraction layer completes the image super-resolution processing on the second image. In this way, the hardware abstraction layer can directly send the reconstructed third image to the application framework layer when the second request is received.

[0103] Alternatively, the receiving time of the second request received by the hardware abstraction layer can be before the hardware abstraction layer completes the image super-resolution processing on the second image. At this time, the hardware abstraction layer suspends responding to the second request when the second request is received; and continues to perform the image super-resolution processing on the second image until the hardware abstraction layer obtains the reconstructed third image, and then responds to the second request.

[0104] In some embodiments, in the case that the electronic device successfully performs the image post-processing on the first image, the hardware abstraction layer sends the processed first image to the application framework layer.

[0105] It can be understood that, in the case that the electronic device successfully performs the image post-processing on the first image, the hardware abstraction layer does not need to process the second image; but directly sends the processed first image to the application framework layer, so that the application framework layer returns the processed first image to the image shooting application.

[0106] After the hardware abstraction layer of the electronic device in the embodiments of the present disclosure receives a first request for requesting a camera to perform a shooting operation, the hardware abstraction layer obtains a first image and a second image associated with the shooting operation; the first image is an original image captured by the camera when performing the shooting operation; the second image is a preview thumbnail image consistent with the image content of the first image. The hardware abstraction layer can send the second image to the application framework layer and perform image post-processing on the first image. Thus, the application framework layer can return the second image to the image shooting application in advance during the image post-processing of the first image, so that the user feels that the shooting speed of the electronic device is improved.

[0107] In addition, in the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image associated with the first image, so as to return a third image obtained by the image super-resolution processing to the application framework layer instead of the first image, thereby reducing the case that the shooting result is lost due to the failure of the post-processing task of the first image, and improving the user experience.

[0108] In some embodiments, the hardware abstraction layer sends the second image to the application framework layer and performs image post-processing on the first image, including:

[0109] The hardware abstraction layer performs image post-processing on the first image through the first thread;

[0110] The second image obtained is sent to the application framework layer through the second thread; the second thread is a thread executed concurrently with the first thread.

[0111] In the embodiments of the present disclosure, the application process corresponding to the image shooting application can include a first thread and a second thread; the second thread is a thread executed concurrently with the first thread.

[0112] It should be noted that a process is a running activity of an application program on a certain data set, and is a basic unit of resource allocation and scheduling of an operating system (for example, an Android system). Each process occupies a block of memory space, and an application program runs on an operating system in the form of one or more processes to implement corresponding functions.

[0113] A thread is an entity of a process, is a basic unit of scheduling and dispatching of a central processing unit (CPU), and is a smaller basic unit that can independently run than a process. The thread itself basically does not have system resources, but only has a little resource (for example, a program counter, a set of registers, or a stack) that is essential in running, but the thread can share all resources of the process with other threads belonging to the process. A thread can create and cancel another thread, and multiple threads in the same process can be executed in parallel.

[0114] In some embodiments, the application process corresponding to the image shooting application can be a camera service process.

[0115] The hardware abstraction layer can execute acquisition of the first image and image post-processing of the first image by using a first thread; and execute acquisition of the second image and sending of the second image by using a second thread.

[0116] Since the first thread and the second thread are two threads that are executed in parallel, the processing process of the first image and the processing process of the second image can be processed in parallel in the process of executing the shooting operation of the electronic device; the processing process of the first image and the processing process of the second image do not affect each other. In other words, even if the first image is lost, the processing process of the second image will not be affected; similarly, even if the second image is lost, the processing process of the first image will not be affected.

[0117] Moreover, by introducing two concurrent threads (i.e., the first thread and the second thread) to concurrently execute the processing of the first image and the second image, the processing of the first image and the processing of the second image have a certain degree of overlap in processing time, thereby shortening the time required for the electronic device to execute the shooting task.

[0118] In some embodiments, the hardware abstraction layer comprises a first abstraction layer and a second abstraction layer.

[0119] The method further comprises:

[0120] The first abstraction layer generates a post-processing task based on the acquired first image, and adds the post-processing task to a task queue;

[0121] The second abstraction layer executes at least one post-processing task in the task queue in sequence;

[0122] In the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image, comprising:

[0123] In a case where the post-processing task for the first image fails, the second abstraction layer deletes the post-processing task for the first image from the task queue.

[0124] The second abstraction layer obtains a second image associated with the first image, and performs image super-resolution processing on the second image to obtain a third image.

[0125] In the embodiments of the present disclosure, the hardware abstraction layer of the electronic device can include a first abstraction layer and a second abstraction layer. It should be noted that the hardware abstraction layer is split into two abstraction layers, i.e., the first abstraction layer and the second abstraction layer in the embodiments of the present disclosure; the first abstraction layer is configured to obtain a first image and a second image captured by a camera from a hardware driver layer; and the processing of the first image and / or the second image is performed by the second abstraction layer. The first abstraction layer obtains the first image captured by the camera from the hardware driver layer, and generates a post-processing task based on the first image.

[0126] It should be noted that the first abstraction layer can generate the post-processing task for the first image based on the shooting parameters carried in the first request.

[0127] After generating the post-processing task for the first image, the first abstraction layer can add the post-processing task to a task queue; and the second abstraction layer can execute the post-processing tasks in the task queue in turn according to the arrangement order of the tasks in the task queue.

[0128] Here, the task queue stores at least one post-processing task.

[0129] In a case where the post-processing task for the first image executed by the second abstraction layer fails, the second abstraction layer deletes the post-processing task for the first image from the task queue.

[0130] It should be noted that the second abstraction layer can process the first image based on a post-processing algorithm; in a case where the algorithm processing fails, it can be determined that the post-processing task for the first image fails. At this time, the second abstraction layer cancels the post-processing task for the first image.

[0131] In some embodiments, in a case where the processing duration of the post-processing task for the first image exceeds a preset duration threshold, it is determined that the post-processing task for the first image fails.

[0132] Here, the duration threshold can be set according to actual needs.

[0133] In the process of executing the post-processing task for the first image by the second abstraction layer, if the processing duration of the post-processing task exceeds the duration threshold, the second abstraction layer can directly determine that the post-processing task for the first image fails; and cancel the post-processing task for the first image.

[0134] After the post-processing task for the first image is canceled, the second abstraction layer obtains a second image associated with the first image, performs image super-resolution processing on the second image to obtain a third image, and sends the third image to the application framework layer, so as to reduce the case that the shooting result is lost due to the failure of the post-processing task for the first image.

[0135] The embodiment of the present disclosure obtains a first image through a first abstraction layer, generates a post-processing task based on the first image, and adds the post-processing task to a task queue to be processed by a second abstraction layer. When the second abstraction layer executes the post-processing task for the first image, if the post-processing task for the first image fails, the second abstraction layer can delete the post-processing task, directly obtain a second image associated with the first image, perform image super-resolution processing on the second image, and return a third image obtained by the image super-resolution processing to the application framework layer instead of the first image, so as to reduce the case that the shooting result is lost due to the failure of the post-processing task for the first image.

[0136] In some embodiments, the method further comprises:

[0137] The first abstraction layer writes the obtained at least one frame of preview image into a preview cache queue;

[0138] The second abstraction layer determines the second image from the preview cache queue according to the first request, and stores the second image in a backup cache area; wherein the images stored in the backup cache area are used for restoring the shooting result in the case that the shooting result of the electronic device is lost.

[0139] In the embodiment of the present disclosure, the preview cache queue is used to store the preview images collected by the camera; so as to facilitate displaying the preview images collected by the camera in real time on the preview interface of the electronic device by calling the preview images in the preview cache queue when the electronic device is shooting.

[0140] When shooting with the electronic device, the hardware driver layer collects the preview images through the camera, and the hardware driver layer sends at least one frame of preview image collected by the camera to the first abstraction layer; the first abstraction layer writes the at least one frame of preview image into the preview cache queue.

[0141] It should be noted that, in the case that the camera collects multiple frames of preview images, the first abstraction layer will sequentially write the multiple frames of images into the preview cache queue according to the collection time of the multiple frames of preview images.

[0142] When the second abstraction layer receives the first request sent by the application framework layer, the second abstraction layer can determine the second image from the preview cache queue according to the first request.

[0143] It should be noted that the first request can carry time information, so that the second abstraction layer determines, according to the time information, a preview image collected at a collection time indicated by the time information from at least one frame of preview images, and determines the preview image as the second image.

[0144] Alternatively, the first request can also carry other information capable of being used to determine the second image, so that the second abstraction layer determines the second image from at least one frame of preview images stored in the preview cache queue according to the information.

[0145] After determining the second image, the second abstraction layer can copy the second image to a backup cache area. Here, the images stored in the backup cache area can at least be used to restore the shooting result in the case that the shooting result of the electronic device is lost.

[0146] It can be understood that, in the case that the post-processing task for the first image fails, the second abstraction layer can directly obtain the second image associated with the first image from the backup cache area; so as to perform image super-resolution processing on the obtained second image to obtain a third image.

[0147] After determining the second image, the second abstraction layer can send the second image to the application framework layer.

[0148] It should be noted that the copying operation of the second image and the sending operation of the second image can be performed simultaneously, or can be performed sequentially, and the execution order of the two operations can be exchanged.

[0149] The second abstraction layer sends the second image to the application framework layer, so that the application framework layer returns the second image to the image shooting application.

[0150] The embodiments of the present disclosure write the obtained at least one frame of preview images into the preview cache queue through the first abstraction layer, which on the one hand enables the application framework layer to realize real-time preview in the shooting process based on the at least one frame of preview images in the preview cache queue. On the other hand, the second abstraction layer can directly determine the first image from the preview cache queue according to the first request, and store the first image to the backup cache area, so that in the case that the image post-processing of the first image fails, the second abstraction layer can directly obtain the second image associated with the first image from the backup cache area, so as to reconstruct a third image based on the second image. In some embodiments, in response to the received image shooting request, the hardware abstraction layer of the terminal obtains the first image and the second image, including:

[0151] In response to the image shooting request, the first abstraction layer obtains scene parameters of a current shooting scene;

[0152] The second abstraction layer determines a shooting strategy corresponding to the scene parameters based on the scene parameters;

[0153] The second abstraction layer sends an image shooting request carrying the shooting strategy to the first abstraction layer;

[0154] The first abstraction layer controls the camera to collect the first image and the second image according to the shooting strategy.

[0155] In the embodiment of the present disclosure, the first abstraction layer receives an image shooting request sent by the application framework layer, and in response to the image shooting request, the first abstraction layer acquires scene parameters of a current shooting scene of the electronic device.

[0156] It can be understood that the first abstraction layer can acquire the scene parameters of the current shooting scene through sensors configured in the electronic device.

[0157] In some embodiments, the image shooting request can carry the scene parameters.

[0158] After acquiring the scene parameters of the current shooting scene of the electronic device, the first abstraction layer can determine a shooting strategy corresponding to the scene parameters based on the scene parameters.

[0159] It should be noted that the determination of the shooting strategy by the first abstraction layer can include switching the camera, setting shooting parameters such as aperture or shutter speed, and the like.

[0160] In some embodiments, the electronic device usually has multiple shooting modes built-in, such as a portrait mode, a landscape mode, and / or a night scene mode, etc. The determination of the shooting strategy corresponding to the scene parameters can include setting a shooting mode matching the scene parameters.

[0161] After determining the shooting mode corresponding to the scene parameters, the first abstraction layer can send an image shooting request carrying the shooting strategy to the second abstraction layer; so that after receiving the image shooting request carrying the shooting strategy, the second abstraction layer can control the camera to shoot according to the shooting strategy, thereby acquiring the first image and the second image.

[0162] In the embodiment of the present disclosure, after receiving the image shooting request, the first abstraction layer can acquire scene parameters of a current shooting scene of the electronic device, so as to determine a shooting strategy for this shooting according to the scene parameters. The first abstraction layer sends an image shooting request carrying the shooting strategy to the second abstraction layer, so that the second abstraction layer controls the camera to shoot according to the shooting strategy corresponding to the current shooting scene, to obtain the first image and the second image, thereby improving the imaging quality of the first image and the second image.

[0163] In some embodiments, the method further includes:

[0164] The application framework layer applies a pre-storage area; the pre-storage area is used to store a shooting image collected when the camera performs a shooting operation;

[0165] The application framework layer sends a second request to the hardware abstraction layer, where the second request carries indication information indicating a storage address of the pre-stored area;

[0166] In response to the received second request, the hardware abstraction layer sends a third image to the application framework layer, including:

[0167] In response to the second request, the hardware abstraction layer stores the third image to the pre-stored area.

[0168] In the embodiments of the present disclosure, the application framework layer can pre-apply the pre-stored area after receiving a shooting instruction initiated by a user through an image shooting application.

[0169] It should be noted that the pre-stored area is used to store a shooting image collected when a camera performs a shooting operation requested by the shooting instruction.

[0170] In some embodiments, the application framework layer can pre-apply the pre-stored area when receiving the second image returned by the hardware abstraction layer.

[0171] It should be noted that, in general, the hardware abstraction layer will apply a storage area for the processed first image after completing the image post-processing of the first image, so that the image shooting application can directly call the first image according to the storage address of the first image.

[0172] Considering that it also takes a certain amount of time to apply a storage area for the first image, in order to shorten the shooting time of the electronic device, the application framework layer can pre-apply the pre-stored area, and after the pre-stored area is applied, the application framework layer sends a second request carrying indication information to the hardware abstraction layer.

[0173] Here, the indication information is used to indicate the storage address of the pre-stored area applied by the application framework layer.

[0174] It should be noted that the shooting time of the electronic device refers to the time required for the process from when the application framework layer initiates an image shooting request in response to a shooting instruction to when the application framework layer receives the first image or the third image sent by the hardware abstraction layer and returns the first image or the third image to the image shooting application.

[0175] The application of the pre-stored area overlaps in time dimension with the processing of the first image and / or the second image, in other words, the application of the pre-stored area and the processing of the first image and / or the second image can be executed concurrently, thus effectively shortening the shooting time of the electronic device.

[0176] In the case that the image post-processing of the first image fails, the hardware abstraction layer receives the second request sent by the application framework layer and stores the reconstructed third image at the storage address indicated by the indication information.

[0177] In some embodiments, in a case where the image post-processing of the first image is successful, the hardware abstraction layer receives a second request sent by the application framework layer, and stores the processed first image in the storage address indicated by the indication information.

[0178] The present disclosure pre-applies a pre-storage area for storing a photographed image by the application framework layer, and carries indication information of a storage address of the pre-storage area in the second request, so that the hardware abstraction layer can directly store the third image in the pre-storage area after receiving the second request, without the hardware abstraction layer applying for a storage area again after obtaining the reconstructed third image. In this way, the photographing time of the electronic device is effectively shortened, and the user experience is provided.

[0179] As shown in Figure 3 , a photographing process of an electronic device according to an exemplary embodiment is shown in Figure 3 . In the camera service system of the electronic device, an asynchronous post-processing unit is introduced to realize asynchronous operation of photographing and post-processing. That is, the hardware abstraction layer returns a small image (i.e., a preview thumbnail) to the application framework layer first, and then returns a large image (i.e., a processed photographed image) asynchronously. Figure 1

[0180] It should be noted that the application framework layer needs to apply for a large image buffer to the APP before returning the large image to the APP, and then the application framework layer sends a large image request to the camera function unit (MockCamera) of the second abstraction layer, which carries the large image buffer. The asynchronous post-processing unit copies the processing result to the large image buffer, and then returns it to the application framework layer by the camera function unit.

[0181] This photographing method can enable the user to see the photographing result (i.e., the small image) faster, thereby improving the user's photographing experience. However, due to various uncertain factors, the asynchronous post-processing unit may fail to process the large image, resulting in loss of the photographing result.

[0182] Therefore, when the asynchronous post-processing unit fails to process the large image (i.e., the processing of the large image fails), the small image can be loaded into the asynchronous post-processing unit, and the small image is subjected to image super-resolution processing by the asynchronous post-processing unit to obtain a processing result. The processing result is used to replace the original large image and return to the APP, so as to reduce the loss of the photographing result.

[0183] As shown in Figure 4 , a photographing process of an electronic device according to an exemplary embodiment is shown in Figure 4 Figure 2 ​​It can be understood that after the user initiates a shooting instruction through the APP, the application framework layer sends a shooting request to the second abstraction layer. The second abstraction layer determines a shooting strategy according to a current shooting scene; and sends the shooting request carrying the shooting strategy to the first abstraction layer.

[0184] The first abstraction layer sends each frame of the preview image obtained to the second abstraction layer; and the second abstraction layer determines a small image from the preview queue and returns the small image to the APP. The second abstraction layer caches the small image to a storage module.

[0185] When the algorithm processing of the large image fails or takes a long time to end due to various uncertain factors, the second abstraction layer cancels the large image processing task; and reads the small image of the current shooting task from the storage module, and loads the small image to a small image super-resolution node of an asynchronous post-processing unit to perform image super-resolution processing on the small image. After the small image is processed by the small image super-resolution node, the large image obtained through the super-resolution processing is returned to the APP through the camera function unit.

[0186] Figure 5 is a block diagram of an image shooting device according to an example embodiment. As shown in Figure 5 , the device mainly includes:

[0187] The acquisition module 501 is configured to, in response to a received first request, acquire, by a hardware abstraction layer of an electronic device, a first image and a second image; wherein the first request is used to request a camera to perform a shooting operation; the first image is an original image obtained based on the camera shooting; and the second image is a preview thumbnail of the first image;

[0188] The first processing module 502 is configured to send, by the hardware abstraction layer, the second image to an application framework layer, and perform image post-processing on the first image;

[0189] The second processing module 503 is configured to, in a case where the image post-processing on the first image fails, perform, by the hardware abstraction layer, image super-resolution processing on the second image to obtain a third image; wherein a resolution of the third image is higher than a resolution of the second image;

[0190] The sending module 504 is configured to, in response to a received second request, send, by the hardware abstraction layer, the third image to the application framework layer; and the second request is used to request to acquire the first image after the image post-processing is completed.

[0191] In some embodiments, the first processing module 502 is configured to:

[0192] The hardware abstraction layer performs the image post-processing on the first image through a first thread;

[0193] send the acquired second image to the application framework layer through a second thread; the second thread is a thread executed concurrently with the first thread.

[0194] In some embodiments, the hardware abstraction layer comprises: a first abstraction layer and a second abstraction layer.

[0195] The first processing module 502 is configured to generate, by the first abstraction layer, a post-processing task based on the acquired first image, and add the post-processing task to a task queue; the second abstraction layer executes at least one post-processing task in the task queue in sequence.

[0196] The second processing module 503 is configured to, in a case where the post-processing task for the first image fails, delete, by the second abstraction layer, the post-processing task for the first image from the task queue; acquire, by the second abstraction layer, a second image associated with the first image, and perform image super-resolution processing on the second image to obtain a third image.

[0197] In some embodiments, the first processing module 502 is configured to:

[0198] The first abstraction layer writes the acquired at least one frame of preview image to a preview cache queue;

[0199] The second abstraction layer determines the second image from the preview cache queue according to the first request, and stores the second image to a backup cache area; the images stored in the backup cache area are used for recovery of a shooting result in a case where the shooting result of the electronic device is lost.

[0200] In some embodiments, the acquisition module 501 is configured to:

[0201] In response to an image shooting request, the first abstraction layer acquires a scene parameter of a current shooting scene;

[0202] The second abstraction layer determines, based on the scene parameter, a shooting strategy corresponding to the scene parameter;

[0203] The second abstraction layer sends, to the first abstraction layer, an image shooting request carrying the shooting strategy;

[0204] The first abstraction layer controls the camera to collect the first image and the second image according to the shooting strategy.

[0205] In some embodiments, the sending module 504 is configured to apply, by the application framework layer, a pre-storage area; the pre-storage area is used for storing a shooting image collected when the camera performs a shooting operation; the application framework layer sends, to the hardware abstraction layer, a second request, wherein the second request carries indication information, and the indication information is used for indicating a storage address of the pre-storage area.

[0206] The sending module 504 is further configured to, in response to the second request, store, by the hardware abstraction layer, the third image to the pre-stored area.

[0207] With regard to the apparatus in the above-described embodiments, the specific manner in which the various modules perform operations has been described in detail in the embodiments related to the method, and thus will not be described in detail here.

[0208] Figure 6 is a structural block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0209] Referring to Figure 6 , the electronic device 600 can include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0210] The processing component 602 usually controls overall operations of the electronic device 600, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 602 can include one or more processors 620 to execute instructions to complete all or part of steps of the above-described methods. In addition, the processing component 602 can include one or more modules so as to facilitate interaction between the processing component 602 and other components. For example, the processing component 602 can include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0211] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include at least one of instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, and videos. The memory 604 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disc, or an optical disc.

[0212] The power component 606 supplies power to various components of the electronic device 600. The power component 606 can include at least one of a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0213] The multimedia component 608 includes a screen providing an output interface between the electronic device 600 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 608 includes a front camera and / or a back camera. When the electronic device 600 is in an operation mode, such as a photographing mode or a video mode, the front camera and / or the back camera can receive external multimedia data. Each of the front camera and the back camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0214] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) that is configured to receive an external audio signal when the electronic device 600 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0215] The I / O interface 612 provides an interface between the processing component 602 and peripheral interface modules, which can include a keyboard, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0216] The sensor component 614 includes one or more sensors for providing status assessments of various aspects of the electronic device 600. For example, the sensor component 614 can detect an open / closed position of the electronic device 600, relative positioning of components, such as a display and a keypad of the electronic device 600, a change in position of the electronic device 600 or a component of the electronic device 600, presence or absence of user contact with the electronic device 600, orientation or acceleration / deceleration of the electronic device 600, and temperature changes of the electronic device 600. The sensor component 614 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 614 can also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, utilized in an imaging application. In some embodiments, the sensor component 614 can further include at least one of an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, and a temperature sensor.

[0217] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an example embodiment, the communication component 616 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 616 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0218] In an example embodiment, the electronic device 600 can be implemented with one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements.

[0219] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 604 including executable instructions or a computer program, is also provided, which can be executed by the processor 620 of the electronic device 600 to complete the above-described method. For example, the non-transitory computer-readable storage medium can be a ROM, a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0220] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform any one of the image capturing methods according to the embodiments of the present disclosure. For example, the method includes:

[0221] In response to the received first request, the hardware abstraction layer of the electronic device acquires the first image and the second image; wherein the first request is used to request the camera to perform a shooting operation; the first image is an original image obtained based on the camera shooting; and the second image is a preview thumbnail of the first image;

[0222] The hardware abstraction layer sends the second image to the application framework layer, and performs image post-processing on the first image;

[0223] In the case that the image post-processing of the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image; wherein the resolution of the third image is higher than that of the second image;

[0224] In response to the received second request, the hardware abstraction layer sends the third image to the application framework layer; the second request is used to request the first image after the image post-processing is completed.

[0225] The embodiment of the disclosure provides a computer program product, which comprises a computer program or executable instructions stored in a computer readable storage medium. The processor of the computer device reads the computer program or executable instructions from the computer readable storage medium, and the processor executes the computer program or executable instructions, so that the computer device executes any one of the image shooting methods provided in the embodiment of the disclosure.

[0226] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The disclosure is intended to cover any variations, uses or adaptations of the disclosure following, in general, the principles of the disclosure and including such features that are evident to those skilled in the art or are known in the art and which are not specifically disclosed. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the appended claims.

[0227] It should be understood that the present disclosure is not limited to the precise structures described above and shown in the drawings and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image capturing method characterized by, The method comprises: In response to a received first request, a hardware abstraction layer of an electronic device acquires a first image and a second image; wherein the first request is used to request a camera to perform a shooting operation; the first image is an original image obtained based on the camera shooting; and the second image is a preview thumbnail of the first image; The hardware abstraction layer sends the second image to an application framework layer and performs image post-processing on the first image; In a case where the image post-processing on the first image fails, the hardware abstraction layer performs image super-resolution processing on the second image to obtain a third image; wherein the resolution of the third image is higher than that of the second image; In response to a received second request, the hardware abstraction layer sends the third image to the application framework layer; the second request is used to request the first image on which the image post-processing is completed.

2. The method of claim 1, wherein, The hardware abstraction layer sends the second image to an application framework layer and performs image post-processing on the first image, comprising: The hardware abstraction layer performs the image post-processing on the first image through a first thread; The hardware abstraction layer sends the acquired second image to the application framework layer through a second thread; wherein the second thread is a thread executed concurrently with the first thread.

3. The method according to claim 1 or 2, characterized in that, The hardware abstraction layer comprises a first abstraction layer and a second abstraction layer; The method further comprises: The first abstraction layer generates a post-processing task based on the acquired first image and adds the post-processing task to a task queue; The second abstraction layer executes at least one post-processing task in the task queue in sequence; In a case where the post-processing task for the first image fails to be executed, the second abstraction layer deletes the post-processing task for the first image from the task queue; The second abstraction layer acquires the second image associated with the first image and performs image super-resolution processing on the second image to obtain the third image. The method further comprises:

4. The method of claim 3, wherein, The first abstraction layer writes the acquired at least one preview image to a preview cache queue; The second abstraction layer determines the second image from the preview cache queue according to a first request and stores the second image to a backup cache area; wherein the images stored in the backup cache area are used to restore the shooting result in a case where the shooting result of the electronic device is lost. The hardware abstraction layer of the terminal acquires the first image and the second image in response to the received image shooting request, comprising:

5. The method of claim 3, wherein, In response to the image shooting request, the first abstraction layer acquires scene parameters of a current shooting scene; The second abstraction layer determines a shooting strategy corresponding to the scene parameters based on the scene parameters; The second abstraction layer sends the image shooting request carrying the shooting strategy to the first abstraction layer; The first abstraction layer controls the camera to collect the first image and the second image according to the shooting strategy. The method further comprises:

6. The method of claim 1, wherein, ​ The application framework layer applies for a pre-storage area; the pre-storage area is used for storing a shooting image collected when the camera performs the shooting operation; The application framework layer sends the second request to the hardware abstraction layer, wherein the second request carries indication information, and the indication information is used to indicate a storage address of the pre-storage area; The hardware abstraction layer sends the third image to the application framework layer in response to the received second request, including: The hardware abstraction layer stores the third image into the pre-storage area in response to the second request.

7. An image capturing apparatus characterized by comprising: Including: An acquisition module is configured to acquire, by a hardware abstraction layer of an electronic device, a first image and a second image in response to a received first request; the first request is used to request a camera to perform a shooting operation; the first image is an original image obtained by the camera shooting; and the second image is a preview thumbnail of the first image; A first processing module is configured to send, by the hardware abstraction layer, the second image to an application framework layer, and perform image post-processing on the first image; A second processing module is configured to perform, by the hardware abstraction layer, image super-resolution processing on the second image to obtain a third image in a case that the image post-processing on the first image fails; the resolution of the third image is higher than that of the second image; A sending module is configured to send, by the hardware abstraction layer, the third image to the application framework layer in response to a received second request; the second request is used to request to acquire the first image on which the image post-processing is completed.

8. An electronic device, comprising: Including: A processor; A memory for storing computer programs or instructions; The processor executes the computer programs or instructions to implement the steps of the method in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing a computer program or instructions, the computer program or instructions comprising the steps of: When the computer programs or instructions in the storage medium are executed by the processor, the steps of the method in any one of claims 1 to 6 are implemented.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer programs or instructions are executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images