Photographing method, electronic device, chip system and storage medium

Abstract

Embodiments of the present application relate to the technical field of terminals, and in particular to a photographing method, an electronic device, a chip system and a storage medium. In the method, the electronic device can obtain different photos on the basis of some identical original images. For example, when the electronic device captures a photo A and a photo B, the electronic device needs to obtain the photo A on the basis of an original image 1 and an original image 2, and the electronic device needs to obtain the photo B on the basis of the original image 2 and an original image 3. Hence, in this method, the same original image may be reused for different photos, thereby improving the frame selection efficiency of the electronic device and saving the storage space of the electronic device.

Description

Photographing method, electronic device, chip system and storage medium

[0001] The present application claims priority from the Chinese patent application No. 202410844508.3 filed on June 26, 2024, and entitled "Photographing method, electronic device, chip system and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of terminals, and in particular to a photographing method, an electronic device, a chip system and a storage medium. BACKGROUND

[0003] With the development of terminal technology, users use electronic devices more and more frequently. In order to meet the needs of users to record and share life anytime and anywhere, electronic devices such as mobile phones and tablets mostly have a photographing function. Electronic devices can take photos and the like through the photographing function. At present, in the process of taking photos by an electronic device, the camera of the electronic device can collect original images, and then the electronic device can obtain photos according to multiple original images, which can improve the quality of the photos taken by the electronic device.

[0004] At present, in the process of taking multiple photos by an electronic device, different photos need to be obtained according to completely different original images. For example, the electronic device takes photo A and photo B, the electronic device needs to obtain photo A according to original image 1 and original image 2, and the electronic device needs to obtain photo B according to original image 3 and original image 4. It can be seen that the electronic device needs at least four original images to obtain two photos, that is, the frame selection efficiency of the electronic device is relatively low. SUMMARY

[0005] Embodiments of the present application provide a photographing method, an electronic device, a chip system and a storage medium. In the method, the electronic device can obtain photos according to part of the same original images, which can improve the frame selection efficiency of the electronic device.

[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a photographing method is provided, which can be applied to an electronic device including a camera, such as a mobile phone. The method comprises: in response to an operation of starting a camera application, the electronic device displays a photographing preview interface. It can be understood that after the electronic device displays the photographing preview interface, the electronic device places raw images captured by the camera into a first queue (e.g., a ZSL queue). The first queue can also be understood as a first storage space, a first storage area, or the like, which is used to carry image data. Next, at a first time point, in response to a first operation of a user, the electronic device obtains a first photo based on k raw images in the first queue; the capture time of the k raw images is before the first time point; k is an integer greater than or equal to 2. The first photographing operation can be a single photographing operation, such as a click operation on a photographing button. Then, at a second time point, in response to a second photographing operation of the user, the electronic device obtains a second photo based on n raw images in the first queue; the capture time of the n raw images is before the second time point, and n is an integer greater than or equal to 2. The second photographing operation can be a single photographing operation, such as a click operation on a photographing button. The k raw images include m raw images of the n raw images; m is an integer greater than or equal to 1, and m is less than n and k.

[0008] It can be seen that in the above method, the electronic device can obtain different photos based on part of the same raw images. For example, the first photo is obtained based on a first raw image and a second raw image, and the second photo is obtained based on the second raw image and a third raw image. In this way, the same raw images can be reused through different photos, the electronic device can obtain multiple photos through a relatively small number of raw images, the frame selection efficiency of the electronic device can be improved, and the storage space of the electronic device can be saved.

[0009] In a possible design of the first aspect, the method further comprises: in response to the operation of starting the camera application, the electronic device displays the photographing preview interface. It can be understood that after the electronic device displays the photographing preview interface, the electronic device places raw images captured by the camera into a first queue (e.g., a ZSL queue). Next, at a third time point, in response to a third photographing operation, a third photo is obtained based on x raw images, and a fourth photo is obtained based on y raw images. The third photographing operation can be a multiple photographing operation, such as a long press operation on a photographing button. x is an integer greater than or equal to 2, and y is an integer greater than or equal to 2. The x raw images include z raw images of the y raw images, z is an integer greater than or equal to 1, and z is less than x and y.

[0010] In this design, the electronic device can obtain different photos according to part of the same original images. For example, the third photo is obtained according to the fourth original image and the fifth original image, and the fourth photo is obtained according to the fifth original image and the sixth original image. In this way, the same original image can be reused through different photos, and the electronic device can obtain multiple photos through a relatively small number of original images, thereby improving the frame selection efficiency of the electronic device. Especially in the scenario of continuous shooting of the electronic device, the electronic device needs to generate multiple photos in a short time. By reusing the same original image through different photos, the electronic device can obtain multiple photos through a relatively small number of original images, thereby improving the photo output efficiency of the electronic device and saving the storage space of the electronic device.

[0011] In a possible design of the first aspect, the k original images are original images that meet a frame selection condition.

[0012] In this design, the electronic device can set the original image as a reference frame after generating a photo according to the original image. For example, after the electronic device obtains a photo according to the first original image and the second original image, the electronic device sets the first original image as a reference frame. In this way, in the process of generating subsequent photos, the electronic device does not generate a photo according to the original image that is a reference frame, which can prevent the photos generated by the electronic device from being confusing, and can improve the quality of the photos obtained by the electronic device.

[0013] In a possible design of the first aspect, the original image that meets the frame selection condition further includes an original image whose difference between the collection time and the operation time of the first shooting operation is less than a shooting time threshold.

[0014] In this design, the electronic device does not obtain a photo according to an original image whose difference between the collection time and the operation time of the first shooting operation is greater than or equal to the shooting time threshold, which can avoid the electronic device shooting a picture that is far away from the shooting time, and can improve the quality of the photos obtained by the electronic device.

[0015] In a possible design of the first aspect, the original image that meets the frame selection condition further includes one or more of the following: an original image that is not a variable exposure frame; an original image that is not an abnormal state; and an original image that is not a switching frame.

[0016] In the case that the electronic device starts the HDR shooting mode, or the environment light intensity of the shooting environment in which the electronic device is located is relatively low, the electronic device can modify the shooting parameter data included in the producer queue, for example, modify the exposure time.

[0017] In a possible design of the first aspect, the electronic device obtains the first photo according to the k frames of raw images in the first queue, including: the electronic device traverses the raw images included in the first queue, and the electronic device determines the raw images meeting the frame selection condition from the raw images included in the first queue. Then, the electronic device puts the raw images meeting the frame selection condition into the second queue. Next, in a case where the number of the raw images included in the second queue is greater than or equal to k frames, the electronic device obtains the first photo according to the raw images included in the second queue.

[0018] In a possible design of the first aspect, the method further includes: in a case where the raw images included in the first queue are all traversed and the number of the raw images included in the second queue is less than k frames, triggering the camera to capture at least two target raw images. Then, the electronic device obtains the photo according to the at least two target raw images.

[0019] In a possible design of the first aspect, the method further includes: in a case where the number of the raw images included in the first queue is less than k frames, the electronic device triggers the camera to capture at least two target raw images. Then, the electronic device obtains the photo according to the at least two target raw images.

[0020] In a possible design of the first aspect, the first queue includes a first sub-queue (for example, a consumer queue) and a second sub-queue (for example, a producer queue), and the electronic device traverses the raw images included in the first queue, including: the electronic device traverses the raw images included in the first sub-queue in a descending order of the capture time. The electronic device traverses the raw images included in the second sub-queue in an ascending order of the capture time.

[0021] In this design, the electronic device traverses the raw images included in the second sub-queue in an ascending order of the capture time, which can improve the speed of the electronic device to capture the photo and reduce the time of see2review.

[0022] In a possible design of the first aspect, the method further includes: using a first image optimization algorithm on the k original images to obtain a fifth photo, the first image optimization algorithm being different from the image optimization algorithm used to obtain the first photo. It can be understood that the electronic device can use the first image optimization algorithm on the k original images to obtain the fifth photo when the electronic device is idle. For example, the mobile phone can store the first original image and the second original image. Then, the mobile phone can obtain the fifth photo according to the first original image and the second original image when the mobile phone is idle. The mobile phone can be idle when the processor occupancy of the mobile phone is lower than an occupancy threshold, or the camera application can be closed, or the mobile phone can be idle for a period of time, which can be obtained by the mobile phone learning the usage habits of the user.

[0023] The mobile phone can use the first image optimization algorithm on the first original image and the second original image to obtain the fifth photo, and the mobile phone can use a second image optimization algorithm on the first original image and the second original image to obtain the first photo. The first image optimization algorithm is different from the second image optimization algorithm. For example, the mobile phone consumes more computing power when running the first image optimization algorithm than when running the second image optimization algorithm.

[0024] It should be understood that, in the process of photographing by the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original images. Therefore, in order to improve the quality of the pictures, such as the aesthetic score of the pictures, the mobile phone can use an image optimization algorithm that consumes more computing power to process the original images to obtain enhanced photos when the mobile phone is idle. In this way, the quality of the photos taken by the mobile phone can be improved.

[0025] In a second aspect, an electronic device is provided, which includes a memory and one or more processors, the memory being coupled to the processors; and wherein the memory stores computer program codes including computer instructions; and when the computer instructions are executed by the processors, the electronic device performs the method provided in the first aspect and any possible design of the first aspect.

[0026] In a third aspect, a computer readable storage medium is provided, which includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device performs the method provided in the first aspect and any possible design of the first aspect.

[0027] In a fourth aspect, a computer program product including instructions is provided, and when the computer program product is executed on an electronic device, the electronic device can perform the method provided in the first aspect and any possible design of the first aspect.

[0028] In a fifth aspect, a chip system is provided, which is applied to an electronic device, and the chip system comprises one or more processors configured to invoke computer instructions to cause the electronic device to perform the method provided in the first aspect and any possible design of the first aspect.

[0029] The technical effects brought by any design of the second aspect to the fifth aspect can be referred to the technical effects brought by different designs of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0030] FIG. 1 is a schematic diagram of generating a photo in some schemes provided by embodiments of the present application;

[0031] FIG. 2 is a schematic diagram of generating a photo in a photographing method provided by embodiments of the present application;

[0032] FIG. 3 is a schematic diagram of a structure of an electronic device provided by embodiments of the present application;

[0033] FIG. 4 is a schematic diagram of an architecture of an electronic device provided by embodiments of the present application;

[0034] FIG. 5 is a schematic diagram of image data flow on an electronic device provided by embodiments of the present application;

[0035] FIG. 6 is a schematic diagram of a flow about a flag setting part in a photographing method provided by embodiments of the present application;

[0036] FIG. 7 is a schematic diagram of a group of user graphical interfaces provided by embodiments of the present application;

[0037] FIG. 8 is a schematic diagram of a flow about a frame selection part in a photographing method provided by embodiments of the present application;

[0038] FIG. 9 is a schematic diagram of a flow about step S701 provided by embodiments of the present application;

[0039] FIG. 10 is a schematic diagram of another structure of an electronic device provided by embodiments of the present application;

[0040] FIG. 11 is a schematic diagram of a structure of a chip system provided by embodiments of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, " / " represents an "or" relationship between the objects before and after the " / ", for example, A / B can represent A or B; in the present application, "and / or" is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions means any combination of the items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second", etc. are used to distinguish the same items or similar items with basically the same function and role. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0042] At the same time, in the embodiments of the present application, the words "exemplary" or "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. On the contrary, the words "exemplary" or "for example" are used to present the relevant concept in a specific way, for the purpose of understanding.

[0043] In the technical solutions disclosed in the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0044] With the development of terminal technology, users use electronic devices more and more frequently. In order to meet the needs of users to record and share life anytime and anywhere, mobile phones, tablets and other electronic devices mostly have a photographing function, and the electronic device can take photos and the like through the photographing function. At present, in the process of photographing photos by the electronic device, the camera of the electronic device can collect original images, and then the electronic device can obtain photos according to multiple original images, which can improve the quality of the photos taken by the electronic device.

[0045] In some solutions, during the process of taking multiple photos by the electronic device, different photos need to be obtained according to different original images. For example, referring to FIG. 1, the electronic device takes photo A and photo B, the electronic device needs to obtain photo A according to original image 1 and original image 2, and the electronic device needs to obtain photo B according to original image 3 and original image 4. It can be seen that the electronic device needs at least 4 frames of original images to obtain two photos, that is, the frame selection efficiency of the electronic device is relatively low.

[0046] Therefore, embodiments of the present application provide a method, in which the electronic device can obtain different photos according to part of the same original images. For example, referring to FIG. 2, the electronic device takes photo A and photo B, the electronic device can obtain photo A according to original image 1 and original image 2, and the electronic device can obtain photo B according to original image 2 and original image 3.

[0047] It can be seen that in this method, the different photos generated by the electronic device can reuse the same original images, so that the electronic device can obtain multiple photos through a relatively small number of original images, and the frame selection efficiency of the electronic device can be improved. At the same time, compared with some solutions, the number of original images required by the photographing method provided by the embodiments of the present application is smaller; for example, in some solutions, four frames of original images are required to obtain two photos, and the photographing method provided by the embodiments of the present application can obtain two photos by using three frames of original images; thus, the storage space of the electronic device can be saved.

[0048] The electronic device can also be referred to as a terminal, a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device 100 can be a mobile phone, a tablet computer, a wearable device, a smart screen, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. with a photographing function. The embodiments of the present application do not make any limitation on the product form of the electronic device.

[0049] Next, the structure of the electronic device is introduced.

[0050] FIG. 3 shows a structural schematic diagram of the electronic device 100, which can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a camera 193, a display screen 194, and the like.

[0051] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0052] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0053] Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0054] The memory in the processor 110 can also be provided for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access, reducing the waiting time of the processor 110, thus improving the efficiency of the system.

[0055] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0056] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the photographing function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0057] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect earphones to play audio through the earphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0058] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0059] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0060] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0061] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc.

[0062] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the algorithm for noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0063] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB, YUV, or the like. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0064] The NPU is a neural-network (NN) computing processor that is configured to quickly process input information by drawing on the structure of a biological neural network, such as by drawing on the transmission mode between neurons in the human brain, and is also configured to continuously self-learn. Through the NPU, the electronic device 100 can implement intelligent cognitive applications, such as image recognition, face recognition, voice recognition, text understanding, and the like.

[0065] The external memory interface 120 can be configured to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video, and the like can be saved in the external memory card.

[0066] The internal memory 121 can be configured to store computer-executable program code including instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, and the like), and the like. The data storage area can store data (such as audio data, a phonebook, and the like) created during use of the electronic device 100, and the like. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0067] After the structure of the electronic device is introduced, the architecture of the electronic device is introduced.

[0068] The software system of the electronic device 100 can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Embodiments of the present application take an Android system with a layered architecture as an example to illustrate the architecture of the electronic device 100.

[0069] By way of example, referring to FIG. 4, the layered architecture divides software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, a hardware abstraction layer (HAL), and a kernel layer. TM

[0070] The application layer can include a series of applications.

[0071] As shown in FIG. 4, the applications can include a camera application, a gallery application, and the like.

[0072] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes a number of pre-defined functions.

[0073] As shown in FIG. 4, the application framework layer can include a window manager, a content provider, a view system, a resource manager, and the like.

[0074] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, take a screenshot, and the like.

[0075] The content provider is used to store and obtain data, and make the data accessible to applications. The data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0076] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and the like. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying a picture.

[0077] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and the like.

[0078] ​The application framework layer can further include a camera interface. The camera interface is used to provide an API to a camera application, so that the camera application can interact with the camera module of the HAL layer by calling the API.

[0079] The hardware abstraction layer is a layer of abstraction between the hardware and the upper layer. It is used to provide a unified interface to the upper layer. The upper layer application does not have to know how the lower layer hardware works, thereby shielding the implementation details of the bottom layer.

[0080] The hardware abstraction layer can provide a standard interface to display the device hardware functions to the higher-level application framework layer. The hardware abstraction layer contains a plurality of library modules, each of which implements an interface for a specific type of hardware component. The library module can include a camera module and the like. When the application framework layer requires access to the device hardware, the system will load the corresponding library module for the hardware component. The manufacturer can define the interface at the hardware abstraction layer.

[0081] The camera module can include a preview channel management component, a photographing channel management component, a Zero-Shutter Lag (ZSL) queue management component, and the like. The preview channel management component can be used to manage the preview channel, the photographing channel management component can be used to manage the photographing channel, and the ZSL management component can be used to manage the ZSL queue.

[0082] In subsequent examples, the functions of the above-mentioned modules will be described in detail.

[0083] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver and a camera driver.

[0084] The following will be described in conjunction with the shooting scene of the electronic device. The working process of the hardware and software of the electronic device during shooting will be described. In FIG. 4, the transmission process of the control instruction in the shooting scene is shown by a dashed line, and the transmission process of the image data in the shooting scene is shown by a solid line.

[0085] After the electronic device starts shooting, such as starting the camera application, the electronic device displays a shooting preview interface. The camera application sends a control instruction to the module of the HAL layer through the camera interface. Next, the preview channel management component sends a control instruction (such as an image acquisition request) to the camera driver. The camera driver acquires an image based on the control instruction through the camera, and sends the image data to the preview channel management component. Then, the preview channel management component sends the image data to the camera application through the camera interface. The camera application displays the image acquired by the camera in real time on the shooting preview interface based on the image data sent by the preview channel management component.

[0086] In the above process, after the camera driver captures an image based on the control instruction, the camera driver can also send the image data to the ZSL queue management component. The ZSL queue management component stores the image data in the ZSL queue. The ZSL queue can be understood as a queue for storing multiple frames of image data, and the ZSL queue can reduce the delay of the electronic device when taking a photo. The ZSL queue management component can manage the ZSL queue according to the first-in first-out rule, and control the ZSL queue to be in a preset length, for example, the image data that enters the ZSL queue first is deleted first by the ZSL queue management module.

[0087] Optionally, in response to the shooting operation in the preview interface, the camera application sends a control instruction indicating taking a photo to the camera module of the HAL layer through the camera interface. Next, the photo taking pipeline management component selects multiple frames of image data from the multiple frames of image data stored in the ZSL queue to generate a photo in response to the control instruction. Next, the photo taking pipeline management component sends the generated photo to the camera application through the camera interface. Next, the camera application can send the photo to the gallery application to store the photo. As can be seen from the description of the process, in this process, the photo taking pipeline management component selects image data from the ZSL queue and generates a photo, so this method can be referred to as ZSL photo taking.

[0088] Optionally, in response to the shooting operation in the preview interface, the camera application sends a control instruction indicating taking a photo to the camera module of the HAL layer through the camera interface. Next, the photo taking pipeline management component sends the control instruction to the camera through the camera driver, and the camera captures image data corresponding to the control instruction based on the control instruction. Then, the photo taking pipeline management component generates a photo based on the image data corresponding to the control instruction. As can be seen from the description of the process, in this process, the photo taking pipeline management component does not select image data from the ZSL queue and generate a photo, but instructs the camera to capture an image and generates a photo based on the image data captured by the camera. Therefore, this method can be referred to as non-ZSL photo taking.

[0089] Comparing ZSL photo taking and non-ZSL photo taking, non-ZSL photo taking needs to perform the process of capturing an image by the camera before generating a photo. The process of capturing an image by the camera can consume time, and thus the electronic device can reduce the see2review time through ZSL photo taking. The see2review time can be understood as the time period from when the user triggers taking a photo to when the electronic device generates a photo taken. For example, in the photo taking preview interface, after clicking to take a photo to generate a thumbnail, immediately click the thumbnail, from the beginning of the thumbnail being completely generated to the time when the photo is loaded in the album.

[0090] That is, the ZSL shooting can enable the electronic device to generate a photo according to an image captured by a camera of the electronic device before the user triggers the shooting; in this way, the shooting time of the electronic device can be reduced.

[0091] Specifically, the detailed description of the preview path management component, the shooting path management component, and the ZSL queue management component can be found below, and will not be described in detail here.

[0092] Before introducing the shooting method provided by the embodiments of the present application, the image data stream (image channel) involved in the embodiments of the present application is introduced.

[0093] For example, referring to FIG. 5, the camera captures an image, and the image captured by the camera is processed by a processing chip related to the camera and can be referred to as image front end (IFE). Then, the mobile phone can obtain encoded image data (such as YUV image data, RGB image data, etc.) and RAW image data according to the image front end. The plurality of encoded image data can form a preview stream, and the image data included in the preview stream is processed by a preview algorithm (such as denoising, zooming, etc.) to be displayed on a preview interface. The ZSL queue management component can obtain RAW image data from the image front end and form a ZSL queue by using the plurality of RAW image data. Then, in response to a shooting operation of the user, the frame selection module selects the plurality of RAW image data from the ZSL queue. Then, the plurality of RAW image data is processed by using a shooting algorithm, such as beautifying, filter, special effect, etc., to generate a photo. The preview stream and the preview algorithm can form a preview path, and the ZSL queue and the shooting algorithm can form a shooting path.

[0094] Optionally, in some embodiments, the camera application can send an image capture request to the camera through the camera interface and the preview pipeline, and the camera can capture an image based on the request to obtain an image corresponding to the request. Then, the mobile phone can obtain a RAW image corresponding to the request and an encoded image corresponding to the request. Next, the ZSL queue management component can place the RAW image corresponding to the request into the ZSL queue. In the case of a user triggering a shooting, the above-mentioned request can carry a shooting event. Then, after the ZSL queue management component places the RAW image corresponding to the request carrying the shooting event into the ZSL queue, the frame selection module of the mobile phone can select one or more RAW images from the ZSL queue and place them into the frame selection queue. The frame selection queue can be understood as a storage space including multiple RAW images, and the RAW images in the storage space are used to generate a photo. Then, the frame sending module of the mobile phone can process the RAW images in the above-mentioned frame selection queue through a shooting algorithm, and the mobile phone can obtain a photo corresponding to the shooting event. The above-mentioned frame selection module and frame sending module can be located in the photo pipeline management component of the HAL layer of the mobile phone, that is, the camera module of the mobile phone can include the photo pipeline management component, and the photo pipeline management component includes the frame selection module and the frame sending module.

[0095] Optionally, in some embodiments, after the mobile phone obtains the RAW image and the encoded image corresponding to the request, the preview pipeline can obtain the encoded image corresponding to the request and display the encoded image corresponding to the request, so that the mobile phone can display the image captured by the camera in real time in the shooting preview interface.

[0096] It can be understood that after receiving the image capture request, it also takes some time for the camera to capture the image, and it also takes some time for the camera to obtain the RAW image after capturing the image. The camera can obtain image data (such as RAW image data) and shooting parameter data (metadata) when capturing the image. The image data and the shooting parameter data correspond to each other, for example, the shooting parameter data obtained based on an image request corresponds to the image data obtained based on the image capture request.

[0097] In some cases, the time point at which the camera obtains the shooting parameter data (metadata) in response to the image capture request can be earlier than the time point at which the camera obtains the RAW image data.

[0098] Optionally, in some embodiments, the ZSL queue can further include a producer queue and a consumer queue. The consumer queue includes the shooting parameter data and the image data corresponding to the shooting parameter data. The producer queue includes the shooting parameter data, and the image data corresponding to the shooting parameter data is not collected by the camera.

[0099] Hereinafter, the technical solutions provided by the embodiments of the present application will be introduced with the example that the electronic device is a mobile phone, the mobile phone has the structure shown in Fig. 3 and the architecture shown in Fig. 4.

[0100] It can be understood that the shooting method provided by the embodiments of the present application can include a flag setting part and a frame selection part.

[0101] Hereinafter, the flag setting part in the shooting method will be introduced first.

[0102] Exemplarily, referring to Fig. 6, the flag setting part in the shooting method provided by the embodiments of the present application can include steps S400-S401. Optionally, the flag setting part can further include one or more steps among steps S402, S403 and S404.

[0103] S400. In response to an operation of starting the camera application, the mobile phone displays a shooting preview interface.

[0104] Exemplarily, referring to Fig. 7, the mobile phone displays a desktop 500, and in response to a trigger operation on a desktop icon 501, the mobile phone displays a shooting preview interface 510. The trigger operation on the desktop icon 501 can be a click operation on the desktop icon 501, or a long press operation on the desktop icon 501, and the like. It should be understood that the trigger operation can also have other designs, and the embodiments of the present application do not make any limitation in this regard.

[0105] The shooting preview interface 510 can include a preview image 511. The preview image 511 is an image collected by the camera of the mobile phone in real time. For example, the image collected by the camera of the mobile phone is sent to the display through a preview channel, and the mobile phone displays the preview image in the shooting preview interface.

[0106] Exemplarily, the mobile phone can display the shooting preview interface in response to a voice instruction of starting the camera application.

[0107] It can be understood that the mobile phone can also display the shooting preview interface in response to other operations, such as an operation of starting the camera application by other applications. Specifically, the trigger condition of triggering the mobile phone to display the shooting preview interface can be designed according to actual use requirements, and the embodiments of the present application do not make any limitation in this regard.

[0108] After the mobile phone executes step S400, the mobile phone can execute step S401.

[0109] S401. In response to the shooting operation, the mobile phone sets a reference frame flag bit for the RAW image.

[0110] The reference flag bit can also be referred to as a shooting reference flag bit, a picture reference flag bit, and the like. Optionally, the flag bit can be recorded in a shooting parameter file (data) corresponding to the RAW image. It should be understood that the RAW image can correspond to at least two files (data), one of which is an image file (data) recording image information, and the other of which is recording some configuration parameters when the camera collects the image, such as automatic focusing setting parameters, white balance setting parameters, automatic exposure setting parameters, image resolution, image bit rate, and the like. Optionally, the RAW image data and the encoded image data obtained from the image collected by the camera at the same time can share a parameter file (data).

[0111] It should be understood that setting the reference frame flag bit for the RAW image can be understood as marking the RAW image as a reference frame. For example, the flag bit A of the RAW image is set to a first value. The flag bit A can indicate whether the frame RAW image is a reference frame by different values. For example, the flag bit A is a first value, such as 1, indicating that the frame RAW image is a reference frame; for another example, the flag bit A is a second value, such as 0, indicating that the frame RAW image is not a reference frame.

[0112] As a possible implementation, the shooting operation can include a single shooting operation or a multiple shooting operation.

[0113] For example, the single shooting operation can be a click operation on the shooting control 512 included in the preview interface 510. For example, the time of touching the touch area corresponding to the shooting control 512 is less than a click time threshold. The click time threshold can be 0.2 seconds, 0.3 seconds, and the like. Alternatively, the single shooting operation can also be a pressing operation on the button of the mobile phone. For example, the time of pressing down the volume adjustment button of the mobile phone is less than a pressing time threshold. The pressing time threshold can be 0.2 seconds, 0.4 seconds, and the like.

[0114] For another example, the multiple shooting operation can be a long press operation on the shooting control 512. For example, the time of touching the touch area corresponding to the shooting control 512 is greater than the click time threshold. Alternatively, the multiple shooting operation can also be a long press operation on the button of the mobile phone. For example, the time of pressing down the volume adjustment button of the mobile phone is greater than the pressing time threshold.

[0115] It should be noted that the above single-shot operation or multi-shot operation is only an example, and in actual application, there can be other more operations to trigger the multi-shot or single-shot of the mobile phone, such as switching the shooting mode of the mobile phone to the continuous shooting mode, and the like. Specifically, the design can be made according to the actual use requirement, and the embodiments of the present application do not limit this.

[0116] As a possible implementation, the mobile phone can select 1 frame of RAW image to set the reference flag bit in the multiple frames of RAW images corresponding to the shooting operation.

[0117] The multiple frames of RAW images corresponding to the shooting operation can be understood as the RAW image selected by the mobile phone for generating a photo in this shooting operation. It can be understood that the reference flag bit corresponds to the photo obtained by the mobile phone. The mobile phone sets the reference flag bit in the RAW image corresponding to the photo obtained by the mobile phone. The single-shot operation corresponds to a group of multiple frames of RAW images, and the multi-shot operation corresponds to multiple groups of multiple frames of RAW images. Optionally, for the related introduction of the multiple frames of RAW images corresponding to the shooting operation, please refer to the introduction of the frame selection part below, which is not repeated here.

[0118] In some embodiments, in response to the shooting operation, the mobile phone sets the reference frame flag bit in the RAW image corresponding to the shooting operation.

[0119] Optionally, the above flag setting part can further include step S402.

[0120] S402. Based on the change of the configuration parameter of the camera, the mobile phone sets the switching frame flag bit in the RAW image.

[0121] The configuration parameter can include one or more of the automatic focusing setting parameter, the white balance setting parameter, the automatic exposure setting parameter, the resolution of the image collected by the camera, and the bit rate of the image collected by the camera.

[0122] It should be understood that the above setting of the switching frame flag bit in the RAW image can be understood as marking the RAW image as a switching frame. For example, the flag bit B of the RAW image is set to a first value. The flag bit B can indicate whether the frame of RAW image is a switching frame by different values. For example, the flag bit B is a first value, such as 1, indicating that the frame of RAW image is a switching frame; for another example, the flag bit B is a second value, such as 0, indicating that the frame of RAW image is not a switching frame.

[0123] As a possible implementation, the mobile phone can set the switching frame flag bit through the image collection request.

[0124] For example, the initial value of the flag bit B is the second value, after the resolution of the image captured by the camera changes, the image capture request can carry the changed resolution. Next, the camera captures the image based on the image capture request carrying the changed resolution, and obtains a RAW image, which is marked as a switching frame. That is, the flag bit B of the RAW image is the first value.

[0125] Optionally, the flag bit setting part can further include step S403.

[0126] S403. Based on the camera capturing an abnormal image, the mobile phone sets the RAW image to an abnormal state.

[0127] Wherein, the camera capturing an abnormal image can include: the camera capturing an image including missing pixel points, or the camera capturing an image including pixel point errors, etc.

[0128] It should be understood that the above setting the RAW image to an abnormal state can be understood as marking the RAW image as abnormal. For example, the flag bit C of the frame RAW image is set to the first value. Wherein, the flag bit C can represent whether the frame RAW image is in an abnormal state through different values. For example, the flag bit C is the first value, such as 1, indicating that the frame RAW image is in an abnormal state, and for example, the flag bit C is the second value, such as 0, indicating that the frame RAW image is not in an abnormal state.

[0129] Optionally, the flag bit setting part can further include step S404.

[0130] S404. Based on the variable exposure capture event, the mobile phone sets the RAW image to a variable exposure frame flag bit.

[0131] In some embodiments, in order to improve the shooting effect of the mobile phone, the mobile phone can generate a variable exposure capture event, so that the camera of the mobile phone captures two or more consecutive frames of images with different exposure times. For example, in order to shoot a high dynamic range imaging (HDR) image, or the ambient light intensity of the shooting environment of the mobile phone is relatively low. The mobile phone can generate a variable exposure capture event.

[0132] Optionally, in response to the variable exposure capture event, the frame sending module modifies the shooting parameter data included in the producer queue, such as modifying the exposure time included in the shooting parameter data. Since the exposure time is modified, the camera will capture image data according to the modified exposure time. Next, after the camera completes the capture of the image data, the image data and the shooting parameter data corresponding to the image data are placed in the consumer queue.

[0133] It should be understood that the above setting the frame flag bit of the RAW image can be understood as marking the RAW image as a variable exposure frame. The variable exposure frame can include a long variable exposure frame or a short variable exposure frame.

[0134] As a possible implementation, in the process of modifying the shooting parameter data included in the frame sending module producer queue described above, the frame sending module can also set a flag bit D of the RAW image corresponding to the shooting parameter data based on the modified exposure time. For example, the flag bit D is a first value indicating that the RAW image is a short variable exposure frame, the flag bit D is a second value indicating that the RAW image is a long variable exposure frame, and the flag bit D is a third value indicating that the RAW image is not a variable exposure frame.

[0135] For example, the initial value of the flag bit D can be the third value, based on the frame sending module modifying the exposure time to the first exposure time, the frame sending module marking the RAW image as a long variable exposure frame; the RAW image is the RAW image corresponding to the shooting parameter whose exposure time is modified by the frame sending module. Based on the frame sending module modifying the exposure time to the second exposure time, the frame sending module marking the RAW image as a short variable exposure frame; the RAW image is the RAW image corresponding to the shooting parameter whose exposure time is modified by the frame sending module. The first exposure time is longer than the second exposure time.

[0136] It can be understood that the above flag bits A, B, C and D can be in the shooting parameter data. That is, the shooting parameter data can include flag bits A, B, C and D.

[0137] It should be pointed out that the above steps S401, S402, S403 and S404 can be executed after step S400. For steps S401, S402, S403 and S404, the execution order of the four can be arbitrary, and the actual use is the criterion, and the embodiments of the present application do not make any limitation on this.

[0138] Next, the frame selection part in the shooting method provided by the embodiments of the present application is introduced.

[0139] For example, referring to FIG. 8, the frame selection part in the shooting method provided by the embodiments of the present application can include steps S700-S702.

[0140] S700. In response to the operation of starting the camera application, the mobile phone displays a shooting preview interface.

[0141] The implementation of this step can refer to the above step S400, which will not be described here.

[0142] S701. In response to the shooting operation, the framing module traverses the RAW images included in the ZSL queue and places the RAW images meeting the framing condition into the framing queue.

[0143] The RAW images meeting the framing condition include one or more of the following: a RAW image that is not a reference frame, a RAW image that is not a variable exposure frame, a RAW image that is not an abnormal state, a RAW image that is not a switching frame, and a RAW image whose acquisition time is less than the shooting time threshold from the operation time of the shooting operation. The shooting time threshold can be 0.3 seconds, 0.2 seconds, or 0.05 seconds, etc. The specific value can be set according to actual use requirements, and the embodiments of the present application do not limit this.

[0144] As a possible implementation, the mobile phone judges the flag bit of the RAW image included in the ZSL queue frame by frame, and places the RAW image meeting the framing condition into the framing queue.

[0145] The mobile phone can acquire the RAW image included in the ZSL queue in the order from late to early according to the acquisition time, and judge the flag bit of the RAW image.

[0146] Optionally, the framing module can judge whether the RAW image is a reference frame through the above-mentioned flag bit A. In the case that the RAW image is not a reference frame, the mobile phone places the RAW image into the framing queue, or the mobile phone judges other flag bits. In the case that the RAW image is a reference frame, the mobile phone performs non-ZSL shooting.

[0147] Optionally, the framing module can judge whether the RAW image is a switching frame through the above-mentioned flag bit B. In the case that the RAW image is not a switching frame, the mobile phone places the RAW image into the framing queue, or the mobile phone judges other flag bits. In the case that the RAW image is a switching frame, the mobile phone performs non-ZSL shooting.

[0148] Optionally, the framing module can judge whether the RAW image is an abnormal state through the above-mentioned flag bit C. In the case that the RAW image is not an abnormal state, the mobile phone places the RAW image into the framing queue, or the mobile phone judges other flag bits. In the case that the RAW image is an abnormal state, the mobile phone judges the next frame of the RAW image.

[0149] Optionally, the framing module can judge whether the RAW image is a variable exposure frame through the above-mentioned flag bit D. In the case that the RAW image is not a variable exposure frame, the mobile phone places the RAW image into the framing queue, or the mobile phone judges other flag bits. In the case that the RAW image is a variable exposure frame, the mobile phone performs non-ZSL shooting.

[0150] Optionally, the frame selection module can determine whether the difference between the original image capture time and the operation time of the shooting operation is less than the shooting time threshold through the timestamp of the original image. In the case where the above difference is less than the shooting time threshold, the mobile phone places the original image into the frame selection queue, or the mobile phone determines other flag bits. In the case where the above difference is greater than or equal to the shooting time threshold, the mobile phone determines the next frame of the original image of the frame of the original image.

[0151] S702. In the case where the number of RAW images included in the frame selection queue is greater than or equal to the preset number, the frame sending module obtains a photo from the RAW images included in the frame selection queue.

[0152] The above preset number is greater than or equal to 1 frame. For example, the above preset number can be 2 frames, 5 frames, etc. In some embodiments, the above preset number can vary according to the processor occupancy of the mobile phone. For example, when the processor occupancy of the mobile phone is relatively high, the mobile phone can reduce the above preset number; when the processor occupancy of the mobile phone is relatively low, the mobile phone can increase the above preset number.

[0153] As a possible implementation, the frame sending module can process the RAW images included in the frame selection queue through image synthesis related algorithms to obtain a photo. The image synthesis related algorithms can include multi-frame synthesis algorithms, such as multi-frame noise reduction (MFNR) algorithms, temporal noise reduction algorithms, rolling shutter correction algorithms, etc. Specifically, the image synthesis related algorithms can be set according to actual use requirements, and the present application does not make any limitation in this regard.

[0154] Optionally, the frame sending module can also select 1 frame of RAW image as a reference frame flag among the RAW images included in the frame selection queue after obtaining a photo from the RAW images included in the frame selection queue. Next, the frame sending module clears the frame selection queue.

[0155] For example, in response to the shooting operation A, the frame selection module places RAW image A, RAW image B and RAW image C into the frame selection queue. Then, the frame sending module obtains photo A from RAW image A, RAW image B and RAW image C included in the frame selection queue. Then, the frame sending module can mark RAW image B as a reference frame according to the relatively high quality of RAW image B. That is, the flag bit A of RAW image B is set to the first value. Next, the frame sending module clears the frame selection queue.

[0156] It can be seen that in the above process, the mobile phone can traverse the RAW images included in the ZSL queue, determine a preset number of RAW images meeting the frame selection condition, and obtain the photo according to the preset number of RAW images meeting the frame selection condition.

[0157] Next, the step S701 is further introduced.

[0158] As a possible implementation, referring to FIG. 9, the step S701 can include steps S900-S909.

[0159] S900. The frame selection module determines the number of RAW images included in the ZSL queue.

[0160] In a case where the number of RAW images included in the ZSL queue is less than the preset number, step S901 is performed.

[0161] In a case where the number of RAW images included in the ZSL queue is greater than or equal to the preset number, step S902 is performed.

[0162] S901. The frame selection module performs non-ZSL shooting.

[0163] The frame selection module performing non-ZSL shooting can be understood as that the frame selection module stops obtaining RAW images from the ZSL queue and does not generate a photo according to the RAW images included in the ZSL queue. For example, the frame selection module can instruct the camera to collect RAW images through a target collection request. Then, the frame selection module generates a photo according to the RAW images collected by the camera based on the target collection request.

[0164] In a case where the number of RAW images included in the ZSL queue is greater than or equal to the preset number, step S902 is performed.

[0165] S902. The frame selection module determines the number of RAW images included in the frame selection queue.

[0166] In a case where the number of RAW images included in the frame selection queue is greater than or equal to the preset number, the step S702 is performed.

[0167] In a case where the number of RAW images included in the frame selection queue is less than the preset number, step S903 is performed.

[0168] S903. The frame selection module obtains the shooting parameter data of the RAW images from the ZSL queue in a first order. The first order can be an order from late to early in collection time. In step S903, the frame selection module obtains RAW images that the frame selection module has not obtained in the process of performing the steps S900-S909 this time.

[0169] Optionally, the ZSL queue includes a consumer queue and a producer queue. The frame selection module obtains the shooting parameter data of the RAW image from the consumer queue according to a first order, and obtains the RAW image from the consumer queue according to a second order. The second order is an order from early to late in the acquisition time. In this implementation manner, since the RAW image corresponding to the shooting parameter data in the producer queue has not been acquired by the camera, the shooting parameter data of the RAW image can be obtained from the producer queue according to the order from early to late in the acquisition time. In this way, when it is confirmed that the RAW image is placed in the frame selection queue, the RAW image can be acquired by the camera more quickly and placed in the frame selection queue. In this way, the speed of generating a photo by the mobile phone can be further improved, and the time of see2review can be further reduced.

[0170] S904. The frame selection module determines whether the target RAW image is before the switching frame.

[0171] The target RAW image can be a RAW image corresponding to the shooting parameter data obtained by the frame selection module in the above step S903.

[0172] As a possible implementation manner, the frame selection module can determine whether the target RAW image is before the switching frame by a value of a flag bit B of the shooting parameter data of a previous frame RAW image of the target RAW image. When the value of the flag bit B of the shooting parameter data of the previous frame RAW image of the target RAW image is a first value, the target RAW image is before the switching frame. When the value of the flag bit B of the shooting parameter data of the previous frame RAW image of the target RAW image is a second value, the target RAW image is after the switching frame.

[0173] In a case where the target RAW image is before the switching frame, the above step S901 is performed.

[0174] In a case where the target RAW image is not before the switching frame, step S905 is performed.

[0175] S905. The frame selection module determines whether the target RAW image is a variable exposure frame.

[0176] As a possible implementation manner, the frame selection module can determine whether the target RAW image is a variable exposure frame by a value of a flag bit D of the shooting parameter data of the target RAW image. When the value of the flag bit D of the target RAW image is a first value or a second value, the target RAW image is a variable exposure frame. When the value of the flag bit D of the target RAW image is a third value, the target RAW image is not a variable exposure frame.

[0177] In a case where the target RAW image is a variable exposure frame, the above step S901 is performed.

[0178] In a case where the target RAW image is not the abnormal state, step S906 is performed.

[0179] S906. The frame selection module determines whether the target RAW image is in an abnormal state.

[0180] As a possible implementation, the frame selection module can determine whether the target RAW image is in the abnormal state by a value of a flag bit C included in the shooting parameter data of the target RAW image. When the value of the flag bit C of the target RAW image is a first value, the target RAW image is in the abnormal state. When the value of the flag bit C of the target RAW image is a second value, the target RAW image is not in the abnormal state.

[0181] In a case where the target RAW image is in the abnormal state, step S900 is performed.

[0182] In a case where the target RAW image is not in the abnormal state, step S907 is performed.

[0183] S907. The frame selection module determines whether a difference between a capture time of the target RAW image and a shooting time is greater than a shooting time threshold.

[0184] The shooting time can be a triggering time of the shooting operation.

[0185] As a possible implementation, the frame selection module can determine whether the difference between the capture time of the target RAW image and the shooting time is greater than the shooting time threshold by a time stamp included in the shooting parameter data of the target RAW image.

[0186] In a case where the frame selection module determines that the difference between the capture time of the target RAW image and the shooting time is greater than the shooting time threshold, step S900 is performed.

[0187] In a case where the frame selection module determines that the difference between the capture time of the target RAW image and the shooting time is less than or equal to the shooting time threshold, step S908 is performed.

[0188] S908. The frame selection module determines whether the target RAW image is a reference frame.

[0189] As a possible implementation, the frame selection module can determine whether the target RAW image is the reference frame by a flag bit A included in the shooting parameter data of the target RAW image. When the value of the flag bit A of the target RAW image is a first value, the target RAW image is the reference frame. When the value of the flag bit A of the target RAW image is a second value, the target RAW image is not the reference frame.

[0190] In a case where the target RAW image is the reference frame, step S900 is performed.

[0191] In the case that the target RAW image is not the reference frame, step S909 is performed.

[0192] S909. The frame selection module puts the target RAW image into the frame selection queue.

[0193] Optionally, in the case that the ZSL queue is traversed and the number of image frames included in the frame selection queue is less than a preset number, step S901 is performed. The ZSL queue being traversed can be understood as that the RAW image with the earliest acquisition time included in the ZSL queue is acquired by the frame selection module and the above judgment is performed.

[0194] In some embodiments, after the mobile phone performs step S702, the mobile phone obtains the photo. The mobile phone can also save the plurality of RAW images obtained for the photo and the shooting parameter data corresponding to each RAW image to a target storage area. The target storage area is different from the storage area corresponding to the ZSL queue and the storage area corresponding to the frame selection queue. The target storage area can be a disk space of the mobile phone or a memory space of the mobile phone, and the embodiments of the present application do not limit this.

[0195] Subsequently, in the case that the mobile phone is in an idle period, the mobile phone generates the enhanced photo by using the image optimization algorithm on the stored plurality of RAW images and the shooting parameter data corresponding to each RAW image.

[0196] The idle period of the mobile phone can be obtained by the mobile phone by analyzing the use habits of the user to the mobile phone. For example, the mobile phone analyzes that the user frequently uses the mobile phone from 8:00 to 18:00 every day, and then the mobile phone obtains that the idle period of the mobile phone is from 18:00 to 8:00 the next day.

[0197] Alternatively, the idle period of the mobile phone can be preset, for example, 0:00-3:00 every day.

[0198] Alternatively, the idle period of the mobile phone can also be obtained by the mobile phone according to the processing occupation. For example, in the case that the processor occupation rate of the mobile phone is less than an occupation rate threshold, the mobile phone generates the enhanced photo by using the image optimization algorithm on the stored plurality of RAW images and the shooting parameter data corresponding to each RAW image.

[0199] Alternatively, the idle period of the mobile phone can also be that the camera application of the mobile phone stops foreground running. For example, in the case that the camera application changes to background running, the mobile phone generates the enhanced photo by using the image optimization algorithm on the stored plurality of RAW images and the shooting parameter data corresponding to each RAW image.

[0200] The technical solutions provided by the embodiments of the present application will be introduced below in combination with the use process of the user to the mobile phone.

[0201] Use process 1

[0202] In response to the operation of starting the camera application, the mobile phone displays a shooting preview interface. It can be understood that after the mobile phone displays the shooting preview interface, the mobile phone places the raw images captured by the camera into a first queue (for example, a ZSL queue). The first queue can also be understood as a first storage space, a first storage area, and the like. The above-mentioned first storage space and first storage area are used to carry image data.

[0203] Next, at a first time point, in response to a first operation of the user, the mobile phone obtains a first photo according to k raw images in the first queue; the capture time of the k raw images is before the first time point; k is an integer greater than or equal to 2. The above-mentioned first shooting operation can be a single shooting operation, such as a click operation on a shooting button.

[0204] After that, at a second time point, in response to a second shooting operation of the user, the mobile phone obtains a second photo according to n raw images in the first queue; the capture time of the n raw images is before the second time point, and n is an integer greater than or equal to 2. The second time point is after the first time point. The above-mentioned second shooting operation can be a single shooting operation, such as a click operation on a shooting button.

[0205] Among them, the k raw images include m raw images in the n raw images; m is an integer greater than or equal to 1, and m is less than n and m is less than k.

[0206] That is, the mobile phone can obtain different photos according to part of the same raw images. For example, the first photo is obtained according to the first raw image and the second raw image, and the second photo is obtained according to the second raw image and the third raw image. In this way, the same raw image can be reused through different photos, and the mobile phone can obtain multiple photos through a smaller number of raw images, which can improve the frame selection efficiency of the mobile phone and save the storage space of the electronic device.

[0207] Optionally, after the mobile phone obtains the first photo, the mobile phone can also store the k raw images constituting the first photo, for example, the mobile phone can store the first raw image and the second raw image. After that, in the case that the mobile phone is idle, the mobile phone can obtain a first enhanced photo according to the above-mentioned first raw image and the second raw image. The above-mentioned case that the mobile phone is idle can be that the processor occupancy of the mobile phone is lower than an occupancy threshold, or can also be that the camera application is closed, or can also be a time period in which the mobile phone is idle, wherein the time period in which the mobile phone is idle can be obtained by the mobile phone learning the use habits of the user to the mobile phone.

[0208] The first image optimization algorithm is different from the second image optimization algorithm. For example, the first image optimization algorithm consumes more computing power than the second image optimization algorithm. It should be understood that, in the process of taking a picture by the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original image. Therefore, in order to improve the quality of the picture, such as the aesthetic score of the picture, the mobile phone can use the image optimization algorithm which consumes more computing power to process the original image to obtain an enhanced picture during the idle time. In this way, the quality of the picture taken by the mobile phone can be improved.

[0209] Optionally, after the mobile phone obtains the first enhanced picture, the mobile phone can replace the first picture with the first enhanced picture. For example, delete the first picture and save the first enhanced picture.

[0210] Process 2

[0211] In response to an operation of starting the camera application, the mobile phone displays a shooting preview interface. It can be understood that, after the mobile phone displays the shooting preview interface, the mobile phone places the original image captured by the camera into a first queue (such as a ZSL queue). The first queue can also be understood as a first storage space, a first storage area, etc. The above-mentioned first storage space and first storage area are used to carry image data.

[0212] Next, at a third time point, in response to a third shooting operation, a third picture is obtained according to x frames of original images, and a fourth picture is obtained according to y frames of original images. The above-mentioned third shooting operation can be multiple shooting operations, such as a long press operation on the shooting button.

[0213] Wherein, x is an integer greater than or equal to 2, y is an integer greater than or equal to 2. The x frames of original images include z frames of original images in the y frames of original images, z is an integer greater than or equal to 1, and z is less than x and y.

[0214] That is, the mobile phone can obtain different photos according to part of the same original images. For example, the third photo is obtained according to the fourth original image and the fifth original image, and the fourth photo is obtained according to the fifth original image and the sixth original image. In this way, the same original image can be reused through different photos, and the mobile phone can obtain multiple photos through a relatively small number of original images, thereby improving the frame selection efficiency of the mobile phone. Especially in the scene of continuous shooting of the mobile phone, the mobile phone needs to generate multiple photos in a short time. By reusing the same original image through different photos, the mobile phone can obtain multiple photos through a relatively small number of original images, thereby improving the photo output efficiency of the mobile phone and saving the storage space of the electronic device.

[0215] Optionally, after the mobile phone obtains the third photo, the mobile phone can also store the x frames of original images constituting the third photo, for example, the mobile phone can store the fourth original image and the fifth original image. Then, in the case that the mobile phone is idle, the mobile phone can obtain a third enhanced photo according to the fourth original image and the fifth original image. Wherein, the above-mentioned case that the mobile phone is idle can be the case that the processor occupancy of the mobile phone is lower than the occupancy threshold, or can also be the case that the camera application is closed, or can also be the idle time period of the mobile phone, wherein the idle time period of the mobile phone can be obtained by the mobile phone learning the use habits of the user.

[0216] Wherein, the mobile phone can use a third image optimization algorithm on the fourth original image and the fifth original image to obtain the third enhanced photo; and the mobile phone can use a fourth image optimization algorithm on the fourth original image and the fifth original image to obtain the third photo. The third image optimization algorithm is different from the fourth image optimization algorithm. For example, the mobile phone consumes more computing power when running the third image optimization algorithm than when running the fourth image optimization algorithm. It should be understood that, in the process of photographing of the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original image. Therefore, in order to improve the quality of the picture, such as the aesthetic score of the picture, the mobile phone can use the image optimization algorithm which consumes more computing power to process the original image to obtain the enhanced photo in the idle time. In this way, the quality of the photo taken by the mobile phone can be improved.

[0217] It should be noted that the personal information used in the technical solutions of the present application is limited to information obtained with the individual consent of the person, including but not limited to informing and reminding the user to read the relevant user agreement (notification) before the user uses the function, and signing the agreement including authorization of relevant user information (authorization).

[0218] The algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments. However, such implementation should not be considered beyond the scope of the present application.

[0219] The present embodiment can divide the functional modules of the electronic device according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be implemented in the form of hardware. It should be noted that the division of the modules in the present embodiment is illustrative and is only a logical functional division. Actual implementation can have another division manner.

[0220] The present embodiment also provides an electronic device, as shown in FIG. 10, which can include one or more processors 1801, a memory 1802 and a communication interface 1803.

[0221] The memory 1802 and the communication interface 1803 are coupled with the processor 1801. For example, the memory 1802, the communication interface 1803 and the processor 1801 can be coupled together through a bus 1804.

[0222] The communication interface 1803 is configured to perform data transmission with other devices. The memory 1802 stores computer program code. The computer program code includes computer instructions, which, when executed by the processor 1801, causes the electronic device to perform the related method steps in the above method embodiments of the present application.

[0223] The processor 1801 can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor can also be a combination that implements computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0224] The bus 1804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 1804 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 10, but it does not mean that there is only one bus or only one type of bus.

[0225] The embodiment of the present application further provides a chip system, as shown in FIG. 11, which comprises at least one processor 2001 and at least one interface circuit 2002. The processor 2001 and the interface circuit 2002 can be interconnected through a line. For example, the interface circuit 2002 can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit 2002 can be used to send signals to other devices (such as the processor 2001). Illustratively, the interface circuit 2002 can read instructions stored in the memory and send the instructions to the processor 2001. When the instructions are executed by the processor 2001, the electronic device can perform various steps in the above embodiment. Of course, the chip system can also include other discrete devices, which are not limited in the embodiment of the present application.

[0226] The embodiment of the present application further provides a computer readable storage medium, which stores computer program codes. When the processor executes the computer program codes, the electronic device performs the related method steps in the above method embodiment.

[0227] The embodiment of the present application further provides a computer program product, which makes the computer execute the related method steps in the above method embodiment when the computer program product runs on the computer.

[0228] The electronic device, the computer readable storage medium or the computer program product provided by the present application are used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0229] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0230] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the modules or units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0231] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or a plurality of physical units, that is, can be located in one place, or can be distributed to a plurality of different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0232] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0233] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for making a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage program codes.

[0234] The above is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A photographing method characterized by comprising: The method is applied to an electronic device including a camera, and the method includes: displaying a preview interface in response to an operation of starting a camera application; placing original images captured by the camera into a first queue after the display of the preview interface; at a first time point, obtaining a first photo from k frames of original images in the first queue in response to a first shooting operation; the k frames of original images are captured before the first time point, and k is an integer greater than or equal to 2; at a second time point, obtaining a second photo from n frames of original images in the first queue in response to a second shooting operation; the second time point is later than the first time point, the n frames of original images are captured before the second time point, and n is an integer greater than or equal to 2; wherein the k frames of original images include m frames of original images in the n frames of original images; m is an integer greater than or equal to 1, and m is less than n and k.

2. The method of claim 1, wherein, The method further includes: at a third time, obtaining a third photo from x frames of original images and a fourth photo from y frames of original images in response to a third shooting operation; x is an integer greater than or equal to 2, and y is an integer greater than or equal to 2; wherein the x frames of original images include z frames of original images in the y frames of original images, z is an integer greater than or equal to 1, and z is less than x and y.

3. The method according to claim 1 or 2, characterized in that, The k frames of original images are k frames of original images meeting a selection condition; the original images meeting the selection condition include original images that are not reference frames.

4. The method of claim 3, wherein, The original images meeting the selection condition further include: original images whose difference between a capture time and an operation time of the first shooting operation is less than a shooting time threshold.

5. The method according to claim 3 or 4, characterized in that, The original images meeting the selection condition further include one or more of: original images that are not variable exposure frames; original images that are not in an abnormal state; original images that are not switching frames.

6. The method according to any one of claims 3-5, characterized in that, The obtaining of the first photo from the k frames of original images in the first queue includes: traversing original images included in the first queue to determine original images meeting the selection condition in the original images included in the first queue; placing the original images meeting the selection condition into a second queue; in a case where a number of original images included in the second queue is greater than or equal to the k frames, obtaining the first photo from the original images included in the second queue.

7. The method of claim 6, wherein, The method further includes: in a case where all original images included in the first queue are traversed and a number of original images included in the second queue is less than the k frames, triggering the camera to capture at least two target original images; obtaining a photo from the at least two target original images.

8. The method of claim 6, wherein, The method further includes, in a case where a number of original images included in the first queue is less than the k frames, triggering the camera to capture at least two target original images; obtaining a photo from the at least two target original images.

9. The method according to any one of claims 6-8, characterized in that, The first queue includes a first subqueue and a second subqueue, and the traversing of the original images included in the first queue includes: traversing the raw images included in the first sub-queue in order from late to early acquisition time; traversing the raw images included in the second sub-queue in order from early to late acquisition time.

10. The method according to any one of claims 1 to 9, characterized in that, The method further comprises: using a first image optimization algorithm on the k frames of raw images to obtain a fifth photo, the first image optimization algorithm being different from the image optimization algorithm used for the first photo.

11. An electronic device, comprising: The electronic device comprises a processor, a memory; the processor is coupled with the memory; the memory is configured to store computer program codes; the computer program codes comprise computer instructions, when the processor executes the computer instructions, the electronic device executes the method as claimed in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises computer instructions, when the computer instructions run on an electronic device, the electronic device executes the method as claimed in any one of claims 1-10.

13. A chip system, characterized by The chip system is applied to an electronic device, and the chip system comprises one or more processors, and the processor is configured to invoke computer instructions to enable the electronic device to execute the method as claimed in any one of claims 1-10.

14. A computer program product, characterised in that, The computer program product comprises instructions, when the computer program product runs on an electronic device, the electronic device executes the method as claimed in any one of claims 1-10.

Images