Shooting method, electronic equipment, chip system and storage medium

Abstract

The embodiment of the invention relates to the technical field of terminals, in particular to a shooting method, electronic equipment, a chip system and a storage medium. In the method, the electronic equipment can obtain different photos according to partially identical original images. For example, the electronic equipment shoots a photo A and a photo B, the electronic equipment needs to obtain the photo A according to the original image 1 and the original image 2, and the electronic equipment needs to obtain the photo B according to the original image 2 and the original image 3. Therefore, in the method, different photos can multiplex the same original image, so that the frame selection efficiency of the electronic equipment can be improved, and the storage space of the electronic equipment can be saved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of terminals, and in particular, to a photographing method, an electronic device, a chip system, and a storage medium. BACKGROUND

[0002] With the development of terminal technology, users use electronic devices more and more frequently. In order to meet the needs of users to record and share life anytime and anywhere, electronic devices such as mobile phones and tablets mostly have a photographing function. Electronic devices can take photos and the like through the photographing function. At present, in the process of taking photos by an electronic device, the camera of the electronic device can collect original images, and then the electronic device can obtain photos according to multiple original images, which can improve the quality of the photos taken by the electronic device.

[0003] At present, in the process of taking multiple photos by an electronic device, different photos need to be obtained according to completely different original images. For example, the electronic device takes photo A and photo B, the electronic device needs to obtain photo A according to original image 1 and original image 2, and the electronic device needs to obtain photo B according to original image 3 and original image 4. It can be seen that the electronic device needs at least four original images to obtain two photos, that is, the frame selection efficiency of the electronic device is relatively low. SUMMARY

[0004] Embodiments of the present application provide a photographing method, an electronic device, a chip system, and a storage medium. In the method, the electronic device can obtain photos according to partially same original images, which can improve the frame selection efficiency of the electronic device.

[0005] To achieve the above object, embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a photographing method is provided, which can be applied to an electronic device including a camera, such as a mobile phone. The method comprises: in response to an operation of starting a camera application, displaying, by the electronic device, a photographing preview interface. It can be understood that after the electronic device displays the photographing preview interface, the electronic device places raw images captured by the camera into a first queue (e.g., a ZSL queue). The first queue can also be understood as a first storage space, a first storage area, or the like, which is used to carry image data. Next, at a first time point, in response to a first operation of a user, the electronic device obtains a first photo according to k raw images in the first queue; the capture time of the k raw images is before the first time point; k is an integer greater than or equal to 2. The first photographing operation can be a single photographing operation, such as a click operation on a photographing button. Then, at a second time point, in response to a second photographing operation of the user, the electronic device obtains a second photo according to n raw images in the first queue; the capture time of the n raw images is before the second time point, and n is an integer greater than or equal to 2. The second photographing operation can be a single photographing operation, such as a click operation on a photographing button. The k raw images include m raw images of the n raw images; m is an integer greater than or equal to 1, and m is less than n and k.

[0007] It can be seen that in the above method, the electronic device can obtain different photos according to part of the same raw images. For example, the first photo is obtained according to a first raw image and a second raw image, and the second photo is obtained according to the second raw image and a third raw image. In this way, the same raw image can be reused through different photos, the electronic device can obtain multiple photos through a small number of raw images, the frame selection efficiency of the electronic device can be improved, and the storage space of the electronic device can be saved.

[0008] In a possible design of the first aspect, the method further comprises: in response to the operation of starting the camera application, displaying, by the electronic device, the photographing preview interface. It can be understood that after the electronic device displays the photographing preview interface, the electronic device places raw images captured by the camera into a first queue (e.g., a ZSL queue). Next, at a third time point, in response to a third photographing operation, a third photo is obtained according to x raw images, and a fourth photo is obtained according to y raw images. The third photographing operation can be a multiple photographing operation, such as a long press operation on a photographing button. x is an integer greater than or equal to 2, and y is an integer greater than or equal to 2. The x raw images include z raw images of the y raw images, z is an integer greater than or equal to 1, and z is less than x and y.

[0009] In this design, the electronic device can obtain different photos according to part of the same original images. For example, the third photo is obtained according to the fourth original image and the fifth original image, and the fourth photo is obtained according to the fifth original image and the sixth original image. In this way, the same original image can be reused through different photos, and the electronic device can obtain multiple photos through a relatively small number of original images, thereby improving the frame selection efficiency of the electronic device. Especially in the scenario of continuous shooting of the electronic device, the electronic device needs to generate multiple photos in a short time. By reusing the same original image through different photos, the electronic device can obtain multiple photos through a relatively small number of original images, thereby improving the photo output efficiency of the electronic device and saving the storage space of the electronic device.

[0010] In a possible design of the first aspect, the k frames of original images are k frames of original images that meet a frame selection condition.

[0011] In this design, the electronic device can set the original image as a reference frame after generating a photo according to the original image. For example, after the electronic device obtains a photo according to the first original image and the second original image, the electronic device sets the first original image as a reference frame. In this way, in the process of generating subsequent photos, the electronic device does not generate a photo according to the original image that is a reference frame, which can prevent the photos generated by the electronic device from being confusing, and can improve the quality of the photos obtained by the electronic device.

[0012] In a possible design of the first aspect, the original image that meets the frame selection condition further includes an original image whose difference between the collection time and the operation time of the first shooting operation is less than a shooting time threshold.

[0013] In this design, the electronic device does not obtain a photo according to an original image whose difference between the collection time and the operation time of the first shooting operation is greater than or equal to the shooting time threshold, which can avoid the electronic device shooting a picture that is far away from the shooting time, and can improve the quality of the photos obtained by the electronic device.

[0014] In a possible design of the first aspect, the original image that meets the frame selection condition further includes one or more of the following: an original image that is not a variable exposure frame; an original image that is not an abnormal state; and an original image that is not a switching frame.

[0015] In the case that the electronic device starts the HDR shooting mode, or the environment light intensity of the shooting environment in which the electronic device is located is relatively low, the electronic device can modify the shooting parameter data included in the producer queue, for example, modify the exposure time.

[0016] In a possible design of the first aspect, the electronic device obtains the first photo according to the k frames of raw images in the first queue, including: the electronic device traverses the raw images included in the first queue, and the electronic device determines the raw images meeting the frame selection condition from the raw images included in the first queue. Then, the electronic device puts the raw images meeting the frame selection condition into the second queue. Next, in a case where the number of the raw images included in the second queue is greater than or equal to k frames, the electronic device obtains the first photo according to the raw images included in the second queue.

[0017] In a possible design of the first aspect, the method further includes: in a case where the raw images included in the first queue are all traversed and the number of the raw images included in the second queue is less than k frames, triggering the camera to capture at least two target raw images. Then, the electronic device obtains the photo according to the at least two target raw images.

[0018] In a possible design of the first aspect, the method further includes: in a case where the number of the raw images included in the first queue is less than k frames, the electronic device triggers the camera to capture at least two target raw images. Then, the electronic device obtains the photo according to the at least two target raw images.

[0019] In a possible design of the first aspect, the first queue includes a first sub-queue (for example, a consumer queue) and a second sub-queue (for example, a producer queue), and the electronic device traverses the raw images included in the first queue, including: the electronic device traverses the raw images included in the first sub-queue in a descending order of the capture time. The electronic device traverses the raw images included in the second sub-queue in an ascending order of the capture time.

[0020] In this design, the electronic device traverses the raw images included in the second sub-queue in an ascending order of the capture time, which can improve the speed of the electronic device to capture the photo and reduce the time of see2review.

[0021] In a possible design of the first aspect, the method further includes: using a first image optimization algorithm on the k original images to obtain a fifth photo, the first image optimization algorithm being different from the image optimization algorithm used to obtain the first photo. It can be understood that the electronic device can use the first image optimization algorithm on the k original images to obtain the fifth photo when the electronic device is idle. For example, the mobile phone can store the first original image and the second original image. Then, the mobile phone can obtain the fifth photo according to the first original image and the second original image when the mobile phone is idle. The mobile phone can be idle when the processor occupancy of the mobile phone is lower than an occupancy threshold, or the camera application can be closed, or the mobile phone can be idle for a period of time, which can be obtained by the mobile phone learning the usage habits of the user.

[0022] For example, the mobile phone can use the first image optimization algorithm on the first original image and the second original image to obtain the fifth photo, and use a second image optimization algorithm on the first original image and the second original image to obtain the first photo. The first image optimization algorithm is different from the second image optimization algorithm. For example, the mobile phone consumes more computing power when running the first image optimization algorithm than when running the second image optimization algorithm.

[0023] It should be understood that, in the process of photographing by the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original images. Therefore, in order to improve the quality of the photo, such as the aesthetic score of the photo, the mobile phone can use an image optimization algorithm that consumes more computing power to process the original images to obtain an enhanced photo when the mobile phone is idle. In this way, the quality of the photo taken by the mobile phone can be improved.

[0024] In a second aspect, an electronic device is provided, which includes a memory and one or more processors, the memory being coupled to the processors; and wherein the memory stores computer program codes including computer instructions; and when the computer instructions are executed by the processors, the electronic device performs the method provided in the first aspect and any possible design of the first aspect.

[0025] In a third aspect, a computer readable storage medium is provided, which includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device performs the method provided in the first aspect and any possible design of the first aspect.

[0026] In a fourth aspect, a computer program product including instructions is provided, and when the computer program product is executed on an electronic device, the electronic device can perform the method provided in the first aspect and any possible design of the first aspect.

[0027] In a fifth aspect, a chip system is provided, which is applied to an electronic device, and the chip system comprises one or more processors configured to invoke computer instructions to cause the electronic device to perform the method provided in the first aspect and any possible design of the first aspect.

[0028] The technical effects brought by any design of the second aspect to the fifth aspect can be referred to the technical effects brought by different designs of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A schematic diagram of generating a photo in some schemes provided by embodiments of the present application;

[0030] Figure 2 A schematic diagram of generating a photo in a photographing method provided by embodiments of the present application;

[0031] Figure 3 A schematic diagram of an electronic device provided by embodiments of the present application;

[0032] Figure 4 A schematic diagram of an architecture of an electronic device provided by embodiments of the present application;

[0033] Figure 5 A schematic diagram of image data flow on an electronic device provided by embodiments of the present application;

[0034] Figure 6 A schematic diagram of a flow about a flag bit setting part in a photographing method provided by embodiments of the present application;

[0035] Figure 7 A schematic diagram of a group of user graphical interfaces provided by embodiments of the present application;

[0036] Figure 8 A schematic diagram of a flow about a frame selection part in a photographing method provided by embodiments of the present application;

[0037] Figure 9 A schematic diagram of a flow about step S701 provided by embodiments of the present application;

[0038] Figure 10 Another schematic diagram of an electronic device provided by embodiments of the present application;

[0039] Figure 11 A schematic diagram of a chip system provided by embodiments of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the present application, unless otherwise specified, " / " represents an "or" relationship between the objects associated before and after, for example, A / B can represent A or B; in the present application, "and / or" is only a description of the associated relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second", etc. are used to distinguish the same items or similar items with basically the same function and role. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0041] At the same time, in the embodiments of the present application, the words "exemplary" or "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. On the contrary, the words "exemplary" or "for example" are used to present the relevant concept in a specific way, for easy understanding.

[0042] In the technical solutions disclosed in the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with the relevant legal regulations and do not violate public order and good customs.

[0043] With the development of terminal technology, users use electronic devices more and more frequently. In order to meet the needs of users to record and share life anytime and anywhere, electronic devices such as mobile phones and tablets mostly have a photographing function, and electronic devices can take photos and the like through the photographing function. At present, in the process of photographing photos by electronic devices, the camera of the electronic device can collect original images, and then the electronic device can obtain photos according to multiple original images, which can improve the quality of the photos taken by the electronic device.

[0044] In some schemes, in the process of taking multiple photos by the electronic device, different photos need to be obtained according to different original images. For example, referring toFigure 1 The electronic device captures photo A and photo B, the electronic device needs to obtain photo A according to original image 1 and original image 2, and the electronic device needs to obtain photo B according to original image 3 and original image 4. It can be seen that the electronic device needs at least 4 frames of original images to obtain two photos, that is, the frame selection efficiency of the electronic device is relatively low.

[0045] Therefore, an embodiment of the present application provides a method, in which the electronic device can obtain different photos according to part of the same original images. For example, referring to Figure 2 The electronic device captures photo A and photo B, the electronic device can obtain photo A according to original image 1 and original image 2, and the electronic device can obtain photo B according to original image 2 and original image 3.

[0046] It can be seen that in this method, the different photos generated by the electronic device can reuse the same original images, so that the electronic device can obtain multiple photos through a relatively small number of original images, and the frame selection efficiency of the electronic device can be improved. At the same time, compared with some schemes, the number of original images required by the photographing method provided by the embodiment of the present application is smaller; for example, in some schemes, four frames of original images are required to obtain two photos, and the photographing method provided by the embodiment of the present application can obtain two photos by using three frames of original images; thereby, the storage space of the electronic device can be saved.

[0047] The electronic device can also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device 100 can be a mobile phone, a tablet computer, a wearable device, a smart screen, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The product form of the electronic device is not limited in the embodiment of the present application.

[0048] Next, the structure of the electronic device is introduced.

[0049] Figure 3A structural diagram of the electronic device 100 is shown, which can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a camera 193, a display screen 194, and the like.

[0050] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0051] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0052] Among them, the controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching and executing instructions.

[0053] The memory in the processor 110 can also be provided for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0054] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0055] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the photographing function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0056] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect earphones to play audio through the earphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0057] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods in the above embodiments.

[0058] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0059] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0060] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc.

[0061] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the algorithm for noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0062] The camera 193 is configured to capture still images or videos. An object projects an optical image through a lens to a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB, YUV, or the like. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0063] The NPU is a neural-network (NN) computing processor that is configured to quickly process input information by drawing on the structure of a biological neural network, such as by drawing on the transmission mode between neurons in the human brain, and is also configured to continuously self-learn. Through the NPU, the electronic device 100 can implement intelligent cognitive applications, such as image recognition, face recognition, voice recognition, text understanding, and the like.

[0064] The external memory interface 120 can be configured to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video, and the like can be saved in the external memory card.

[0065] The internal memory 121 can be configured to store computer-executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, and the like), and the like. The data storage area can store data (such as audio data, a phonebook, and the like) created during use of the electronic device 100, and the like. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0066] After the structure of the electronic device is introduced, the architecture of the electronic device is introduced.

[0067] The software system of the electronic device 100 can employ a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. Embodiments of the present application take an Android system with a layered architecture as an example to illustrate the architecture of the electronic device 100.

[0068] For example, referring to Figure 4 , a layered architecture divides software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android TM system is divided into four layers, from top to bottom, the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0069] The application layer can include a series of applications.

[0070] As shown in Figure 4 , the applications can include a camera application, a gallery application, and the like.

[0071] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0072] As shown in Figure 4 , the application framework layer can include a window manager, a content provider, a view system, a resource manager, and the like.

[0073] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and take screenshots, etc.

[0074] The content provider is used to store and obtain data, and make the data accessible to applications. The data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, and the like.

[0075] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and the like. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0076] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and the like.

[0077] The application framework layer can further include a camera interface. The camera interface is used to provide an API to a camera application, so that the camera application can interact with the camera module of the HAL layer by calling the API.

[0078] The hardware abstraction layer is a layer of abstraction between the hardware and the upper layer. It is used to provide a unified interface to the upper layer. The upper layer application does not have to know how the lower layer hardware works, thereby shielding the implementation details of the bottom layer.

[0079] The hardware abstraction layer can provide a standard interface to display the device hardware functions to the higher-level application framework layer. The hardware abstraction layer includes a plurality of library modules, each of which implements an interface for a specific type of hardware component. The library module can include a camera module and the like. When the application framework layer requires access to the device hardware, the system will load the corresponding library module for the hardware component. The manufacturer can define the interface at the hardware abstraction layer.

[0080] The camera module can include a preview channel management component, a photographing channel management component, a Zero-Shutter Lag (ZSL) queue management component, and the like. The preview channel management component can be used to manage the preview channel, the photographing channel management component can be used to manage the photographing channel, and the ZSL management component can be used to manage the ZSL queue.

[0081] In subsequent examples, the functions of the above-mentioned modules will be described in detail.

[0082] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver and a camera driver.

[0083] The working process of the hardware and software of the electronic device during the photographing process will be described in the photographing scenario of the electronic device. Figure 4 In the photographing scenario, the transmission process of the control instruction is shown by a dashed line, and the transmission process of the image data is shown by a solid line.

[0084] After the electronic device starts photographing, such as starting the camera application, the electronic device displays a photographing preview interface. The camera application sends a control instruction to the module of the HAL layer through the camera interface. Next, the preview channel management component sends a control instruction (such as an image acquisition request) to the camera driver. The camera driver acquires an image based on the control instruction, and sends the image data to the preview channel management component. Then, the preview channel management component sends the image data to the camera application through the camera interface. The camera application displays the image acquired by the camera driver in real time on the photographing preview interface based on the image data sent by the preview channel management component.

[0085] In the above process, after the camera driver captures an image based on the control instruction, the camera driver can also send the image data to the ZSL queue management component. The ZSL queue management component stores the image data in the ZSL queue. The ZSL queue can be understood as a queue for storing multiple frames of image data, and the ZSL queue can reduce the delay of the electronic device when taking a photo. The ZSL queue management component can manage the ZSL queue according to the first-in first-out rule, and control the ZSL queue to be in a preset length, for example, the image data that enters the ZSL queue first is deleted first by the ZSL queue management module.

[0086] Optionally, in response to the shooting operation in the preview interface, the camera application sends a control instruction indicating taking a photo to the camera module of the HAL layer through the camera interface. Next, the photo taking pipeline management component selects multiple frames of image data from the multiple frames of image data stored in the ZSL queue to generate a photo in response to the control instruction. Next, the photo taking pipeline management component sends the generated photo to the camera application through the camera interface. Next, the camera application can send the photo to the gallery application to store the photo. As can be seen from the description of the process, in this process, the photo taking pipeline management component selects image data from the ZSL queue and generates a photo, so this method can be referred to as ZSL photo taking.

[0087] Optionally, in response to the shooting operation in the preview interface, the camera application sends a control instruction indicating taking a photo to the camera module of the HAL layer through the camera interface. Next, the photo taking pipeline management component sends the control instruction to the camera through the camera driver, and the camera captures image data corresponding to the control instruction based on the control instruction. Then, the photo taking pipeline management component generates a photo based on the image data corresponding to the control instruction. As can be seen from the description of the process, in this process, the photo taking pipeline management component does not select image data from the ZSL queue and generate a photo, but instructs the camera to capture an image and generates a photo based on the image data captured by the camera. Therefore, this method can be referred to as non-ZSL photo taking.

[0088] Comparing ZSL photo taking and non-ZSL photo taking, non-ZSL photo taking needs to perform the process of capturing an image by the camera before generating a photo. The process of capturing an image by the camera can consume time, and thus the electronic device can reduce the see2review time through ZSL photo taking. The see2review time can be understood as the time period from when the user triggers taking a photo to when the electronic device generates a photo taken. For example, in the photo taking preview interface, after clicking to take a photo to generate a thumbnail, immediately click the thumbnail, from the beginning of the thumbnail being completely generated to the time when the photo is loaded in the album.

[0089] That is, the ZSL shooting can enable the electronic device to generate a photo according to an image captured by a camera of the electronic device before the user triggers the shooting; in this way, the shooting time of the electronic device can be reduced.

[0090] Specifically, the detailed description of the preview path management component, the shooting path management component, and the ZSL queue management component can be found below, and will not be described in detail here.

[0091] Before introducing the shooting method provided by the embodiments of the present application, the image data stream (image channel) involved in the embodiments of the present application is introduced.

[0092] For example, referring to Figure 5 , the camera captures an image, and the image captured by the camera is processed by a processing chip related to the camera and can be referred to as an image front end (IFE). Then, the mobile phone can obtain encoded image data (such as YUV image data, RGB image data, etc.) and RAW image data according to the image front end. The plurality of encoded image data can form a preview stream, and the image data included in the preview stream is processed by a preview algorithm (such as denoising, zooming, etc.) to be displayed on a preview interface. The ZSL queue management component can obtain RAW image data from the image front end and form a ZSL queue by using the plurality of RAW image data. Then, in response to a shooting operation of the user, the frame selection module selects the plurality of RAW image data from the ZSL queue. Then, the plurality of RAW image data is processed by using a shooting algorithm, such as beautifying, filter, special effect, etc., to generate a photo. The preview stream and the preview algorithm can form a preview path, and the ZSL queue and the shooting algorithm can form a shooting path.

[0093] Optionally, in some embodiments, the camera application can send an image capture request to the camera through the camera interface and the preview pipeline, and the camera can capture an image based on the request to obtain an image corresponding to the request. Then, the mobile phone can obtain a RAW image corresponding to the request and an encoded image corresponding to the request. Next, the ZSL queue management component can place the RAW image corresponding to the request into the ZSL queue. In the case of a user triggering a shooting, the above-mentioned request can carry a shooting event. Then, after the ZSL queue management component places the RAW image corresponding to the request carrying the shooting event into the ZSL queue, the frame selection module of the mobile phone can select one or more RAW images from the ZSL queue and place them into the frame selection queue. The frame selection queue can be understood as a storage space including multiple RAW images, and the RAW images in the storage space are used to generate a photo. Then, the frame sending module of the mobile phone can process the RAW images in the above-mentioned frame selection queue through a shooting algorithm, and the mobile phone can obtain a photo corresponding to the shooting event. The above-mentioned frame selection module and frame sending module can be located in the photo pipeline management component of the HAL layer of the mobile phone, that is, the camera module of the mobile phone can include the photo pipeline management component, and the photo pipeline management component includes the frame selection module and the frame sending module.

[0094] Optionally, in some embodiments, after the mobile phone obtains the RAW image and the encoded image corresponding to the request, the preview pipeline can obtain the encoded image corresponding to the request and display the encoded image corresponding to the request, so that the mobile phone can display the image captured by the camera in real time in the shooting preview interface.

[0095] It can be understood that after receiving the image capture request, it also takes some time for the camera to capture the image, and it also takes some time for the camera to obtain the RAW image after capturing the image. The camera can obtain image data (such as RAW image data) and shooting parameter data (metadata) when capturing the image. The image data and the shooting parameter data correspond to each other, for example, the shooting parameter data obtained based on an image request corresponds to the image data obtained based on the image capture request.

[0096] In some cases, the time point at which the camera obtains the shooting parameter data (metadata) in response to the image capture request can be earlier than the time point at which the camera obtains the RAW image data.

[0097] Optionally, in some embodiments, the ZSL queue can further include a producer queue and a consumer queue. The consumer queue includes the shooting parameter data and the image data corresponding to the shooting parameter data. The producer queue includes the shooting parameter data, and the image data corresponding to the shooting parameter data is not collected by the camera.

[0098] Hereinafter, taking the electronic device as a mobile phone, the mobile phone having the above-mentioned Figure 3 structure and the above-mentioned Figure 4 architecture as examples, the technical solutions provided by the embodiments of the present application are introduced.

[0099] It can be understood that the shooting method provided by the embodiments of the present application can include a flag setting part and a frame selection part.

[0100] Hereinafter, the flag setting part in the shooting method is introduced first.

[0101] For example, referring to Figure 6 , the flag setting part in the shooting method provided by the embodiments of the present application can include steps S400-S401. Optionally, the flag setting part can further include one or more steps among steps S402, S403 and S404.

[0102] S400. In response to the operation of starting the camera application, the mobile phone displays a shooting preview interface.

[0103] For example, referring to Figure 7 , the mobile phone displays a desktop 500, and in response to a trigger operation on a desktop icon 501, the mobile phone displays a shooting preview interface 510. The trigger operation on the desktop icon 501 can be a click operation on the desktop icon 501, or a long press operation on the desktop icon 501, and the like. It should be understood that the trigger operation can also have other designs, and the embodiments of the present application do not make any limitation in this regard.

[0104] The shooting preview interface 510 can include a preview image 511. The preview image 511 is an image collected by the camera of the mobile phone in real time. For example, the image collected by the camera of the mobile phone is sent to the display through a preview channel, and the mobile phone displays the preview image in the shooting preview interface.

[0105] For example, the mobile phone can display the shooting preview interface in response to a voice instruction of starting the camera application.

[0106] It can be understood that the mobile phone can also respond to other operations, such as the operation of starting the camera application by other applications, and display the shooting preview interface. Specifically, it can be designed according to the actual use requirement, and the present application embodiment does not make any limitation on the trigger condition of triggering the mobile phone to display the shooting preview interface.

[0107] After the mobile phone executes step S400, the mobile phone can execute step S401.

[0108] S401. In response to the shooting operation, the mobile phone sets a reference frame flag bit for the RAW image.

[0109] The reference flag bit can also be referred to as a shooting reference flag bit, a picture reference flag bit, etc. Optionally, the flag bit can be recorded in the shooting parameter file (data) corresponding to the RAW image. It should be understood that the RAW image can correspond to at least two files (data), one is an image file (data) recording image information, and the other is recording some configuration parameters when the camera collects the image, such as automatic focusing setting parameters, white balance setting parameters, automatic exposure setting parameters, image resolution, image bit rate, etc. Optionally, the RAW image data and the encoded image data obtained by the camera collecting the image at the same time can share one parameter file (data).

[0110] It should be understood that the above setting of the reference frame flag bit for the RAW image can be understood as marking the RAW image as a reference frame. For example, the flag bit A of the RAW image is set to a first value. The flag bit A can indicate whether the frame RAW image is a reference frame by different values. For example, the flag bit A is a first value, such as 1, indicating that the frame RAW image is a reference frame; for example, the flag bit A is a second value, such as 0, indicating that the frame RAW image is not a reference frame.

[0111] As a possible implementation, the above shooting operation can include single shooting operation or multiple shooting operations.

[0112] For example, the touch operation of the touch control area corresponding to the shooting control 512 is less than the click time threshold; wherein the click time threshold can be 0.2 seconds, 0.3 seconds, etc. Alternatively, the single shooting operation can also be the pressing operation of the button of the mobile phone. For example, the time of pressing down the volume adjustment button of the mobile phone is less than the pressing time threshold; wherein the pressing time threshold can be 0.2 seconds, 0.4 seconds, etc.

[0113] For example, the multi-shot operation can be a long press operation on the photographing control 512. For example, the time for touching the touch area corresponding to the photographing control 512 is greater than a click time threshold. Alternatively, the multi-shot operation can also be a long press operation on a button of the mobile phone. For example, the time for pressing a volume adjustment button of the mobile phone is greater than a press time threshold.

[0114] It should be noted that the single-shot operation or the multi-shot operation described above are only examples, and in actual applications, other more operations can trigger the mobile phone to take multiple or single shots. For example, switching the photographing mode of the mobile phone to a continuous shooting mode, and the like. Specifically, the design can be made according to actual use requirements, and the embodiments of the present application do not limit this.

[0115] As a possible implementation, the mobile phone can select one frame of RAW image to set a reference flag bit in the multiple frames of RAW images corresponding to the photographing operation.

[0116] It can be understood that the multiple frames of RAW images corresponding to the photographing operation are the RAW images selected by the mobile phone for generating a photo in this photographing operation. It can be understood that the reference flag bit corresponds to the photo taken by the mobile phone. The mobile phone sets the reference flag bit in the RAW image corresponding to the photo taken by the mobile phone. The single-shot operation corresponds to a group of multiple frames of RAW images, and the multi-shot operation corresponds to multiple groups of multiple frames of RAW images. Optionally, for the related introduction of the multiple frames of RAW images corresponding to the photographing operation, please refer to the introduction of the frame selection part below, which is not repeated here.

[0117] In some embodiments, in response to the photographing operation, the mobile phone sets a reference frame flag bit in the RAW image corresponding to the photographing operation.

[0118] Optionally, the flag setting part can further include step S402.

[0119] S402. The mobile phone sets a switching frame flag bit in the RAW image based on a change in the configuration parameter of the camera.

[0120] The configuration parameter can include one or more of an automatic focusing setting parameter, a white balance setting parameter, an automatic exposure setting parameter, a resolution of an image captured by the camera, and a bit rate of the image captured by the camera.

[0121] It should be understood that the above setting the switching frame flag for the RAW image can be understood as marking the RAW image as a switching frame. For example, the flag B of the RAW image is set to a first value. The flag B can represent whether the RAW image is a switching frame by different values. For example, the flag B is a first value, such as 1, indicating that the RAW image is a switching frame; for another example, the flag B is a second value, such as 0, indicating that the RAW image is not a switching frame.

[0122] As a possible implementation, the mobile phone can set the switching frame flag through an image acquisition request.

[0123] For example, the initial value of the flag B is a second value, and after the resolution of the image captured by the camera changes, the image acquisition request can carry the changed resolution. Next, the camera captures an image based on the image acquisition request carrying the changed resolution to obtain a RAW image, which is marked as a switching frame. That is, the flag B of the RAW image is a first value.

[0124] Optionally, the above flag setting part can further include step S403.

[0125] S403. Based on the camera capturing an abnormal image, the mobile phone sets the RAW image to an abnormal state.

[0126] The camera capturing an abnormal image can include the camera capturing an image including missing pixel points, or the camera capturing an image including pixel point errors, and the like.

[0127] It should be understood that the above setting the RAW image to an abnormal state can be understood as marking the RAW image as abnormal. For example, the flag C of the RAW image is set to a first value. The flag C can represent whether the RAW image is in an abnormal state by different values. For example, the flag C is a first value, such as 1, indicating that the RAW image is in an abnormal state, and for another example, the flag C is a second value, such as 0, indicating that the RAW image is not in an abnormal state.

[0128] Optionally, the above flag setting part can further include step S404.

[0129] S404. Based on the variable exposure acquisition event, the mobile phone sets the RAW image to a variable exposure frame flag.

[0130] In some embodiments, in order to improve the shooting effect of the mobile phone, the mobile phone can generate a variable exposure capture event, so that the camera of the mobile phone captures two or more consecutive images with different exposure times. For example, the mobile phone generates a variable exposure capture event in order to capture a high dynamic range imaging (HDR) image, or the ambient light intensity of the shooting environment of the mobile phone is relatively low.

[0131] Optionally, in response to the variable exposure capture event, the frame sending module modifies the shooting parameter data included in the producer queue, such as modifying the exposure time included in the shooting parameter data. Since the exposure time is modified, the camera will capture image data according to the modified exposure time. Next, after the camera completes the capture of the image data, the image data and the shooting parameter data corresponding to the image data are placed in the consumer queue.

[0132] It should be understood that the above setting of the frame flag bit of the RAW image can be understood as marking the RAW image as a variable exposure frame. The variable exposure frame can include a long variable exposure frame or a short variable exposure frame.

[0133] As a possible implementation, in the process of modifying the shooting parameter data included in the producer queue by the frame sending module, the frame sending module can also set the flag bit D of the RAW image corresponding to the shooting parameter data based on the modified exposure time. For example, the flag bit D is a first value indicating that the RAW image is a short variable exposure frame, the flag bit D is a second value indicating that the RAW image is a long variable exposure frame, and the flag bit D is a third value indicating that the RAW image is not a variable exposure frame.

[0134] For example, the initial value of the flag bit D can be the third value, and based on the modification of the exposure time by the frame sending module to the first exposure time, the frame sending module marks the RAW image as a long variable exposure frame. The RAW image is the RAW image corresponding to the shooting parameter whose exposure time is modified by the frame sending module. Based on the modification of the exposure time by the frame sending module to the second exposure time, the frame sending module marks the RAW image as a short variable exposure frame. The RAW image is the RAW image corresponding to the shooting parameter whose exposure time is modified by the frame sending module. The first exposure time is longer than the second exposure time.

[0135] It can be understood that the above flag bits A, B, C and D can be included in the shooting parameter data. That is, the shooting parameter data can include the flag bits A, B, C and D.

[0136] It should be noted that the above steps S401, S402, S403 and S404 can be executed after step S400. For steps S401, S402, S403 and S404, the execution order of the four can be arbitrary, and the actual use case is used as the criterion, and the embodiments of the present application do not make any limitation on this.

[0137] Next, the frame selection part in the photographing method provided by the embodiments of the present application is introduced.

[0138] For example, referring to Figure 8 The frame selection part in the photographing method provided by the embodiments of the present application can include steps S700-S702.

[0139] S700. In response to the operation of starting the camera application, the mobile phone displays a photographing preview interface.

[0140] The implementation of this step can refer to the above step S400, which will not be repeated here.

[0141] S701. In response to the photographing operation, the frame selection module traverses the RAW images included in the ZSL queue and puts the RAW images meeting the frame selection condition into the frame selection queue.

[0142] The RAW images meeting the frame selection condition include one or more of the following: an original image that is not a reference frame, an original image that is not a variable exposure frame, an original image that is not in an abnormal state, an original image that is not a switching frame, and an original image whose time difference between the acquisition time and the operation time of the photographing operation is less than a photographing time threshold. The photographing time threshold can be 0.3 seconds, 0.2 seconds, 0.05 seconds, etc., and can be set according to actual use requirements, which is not limited by the embodiments of the present application.

[0143] As a possible implementation, the mobile phone judges the flag bit of the RAW images included in the ZSL queue frame by frame, and puts the RAW images meeting the frame selection condition into the frame selection queue.

[0144] The mobile phone can acquire the RAW images included in the ZSL queue in the order from late to early according to the acquisition time, and judge the flag bit of the RAW image.

[0145] Optionally, the frame selection module can judge whether the original image is a reference frame through the above flag bit A. In the case that the original image is not a reference frame, the mobile phone puts the original image into the frame selection queue, or the mobile phone judges other flag bits. In the case that the original image is a reference frame, the mobile phone performs non-ZSL photographing.

[0146] Optionally, the frame selection module can determine whether the original image is a switching frame according to the flag B. In the case that the original image is not a switching frame, the mobile phone puts the original image into the frame selection queue, or the mobile phone determines other flags. In the case that the original image is a switching frame, the mobile phone performs non-ZSL shooting.

[0147] Optionally, the frame selection module can determine whether the original image is an abnormal state according to the flag C. In the case that the original image is not an abnormal state, the mobile phone puts the original image into the frame selection queue, or the mobile phone determines other flags. In the case that the original image is an abnormal state, the mobile phone determines the next original image of the original image.

[0148] Optionally, the frame selection module can determine whether the original image is a variable exposure frame according to the flag D. In the case that the original image is not a variable exposure frame, the mobile phone puts the original image into the frame selection queue, or the mobile phone determines other flags. In the case that the original image is a variable exposure frame, the mobile phone performs non-ZSL shooting.

[0149] Optionally, the frame selection module can determine whether the difference between the original image acquisition time and the operation time of the shooting operation is less than the shooting time threshold according to the time stamp of the original image. In the case that the difference is less than the shooting time threshold, the mobile phone puts the original image into the frame selection queue, or the mobile phone determines other flags. In the case that the difference is greater than or equal to the shooting time threshold, the mobile phone determines the next original image of the original image.

[0150] S702. In the case that the number of RAW images included in the frame selection queue is greater than or equal to the preset number, the frame sending module obtains a photo according to the RAW images included in the frame selection queue.

[0151] The preset number is greater than or equal to 1 frame. For example, the preset number can be 2 frames, 5 frames, etc. In some embodiments, the preset number can change according to the processor occupancy rate of the mobile phone. For example, when the processor occupancy rate of the mobile phone is relatively high, the mobile phone can reduce the preset number; when the processor occupancy rate of the mobile phone is relatively low, the mobile phone can increase the preset number.

[0152] As a possible implementation, the frame sending module can process the RAW images included in the frame selection queue by image synthesis related algorithms to obtain the photo. The image synthesis related algorithms can include multi-frame synthesis algorithms, such as multi-frame noise reduction (MFNR) algorithm, temporal noise reduction algorithm, rolling shutter correction algorithm, etc. Specifically, the image synthesis related algorithms can be set according to actual use requirements, and the embodiments of the present application do not make any limitation.

[0153] Optionally, after obtaining the photo according to the RAW images included in the frame selection queue, the frame sending module can select one RAW image from the RAW images included in the frame selection queue to set the reference frame flag. Next, the frame sending module clears the frame selection queue.

[0154] For example, in response to the shooting operation A, the frame selection module puts the RAW image A, the RAW image B and the RAW image C into the frame selection queue. Then, the frame sending module obtains the photo A according to the RAW image A, the RAW image B and the RAW image C included in the frame selection queue. Then, the frame sending module can mark the RAW image B as the reference frame according to the high quality of the RAW image B. That is, the flag A of the RAW image B is set to the first value. Next, the frame sending module clears the frame selection queue.

[0155] It can be seen that in the above process, the mobile phone can traverse the RAW images included in the ZSL queue, determine a preset number of RAW images meeting the frame selection condition, and obtain the photo according to the preset number of RAW images meeting the frame selection condition.

[0156] Next, the step S701 will be further introduced.

[0157] As a possible implementation, referring to Figure 9 , the step S701 can include steps S900-S909.

[0158] S900. The frame selection module determines the number of RAW images included in the ZSL queue.

[0159] In the case that the number of RAW images included in the ZSL queue is less than the preset number, step S901 is performed.

[0160] In the case that the number of RAW images included in the ZSL queue is greater than or equal to the preset number, step S902 is performed.

[0161] S901. The frame selection module performs non-ZSL shooting.

[0162] In the case that the number of RAW images included in the ZSL queue is greater than or equal to the preset number, step S902 is performed.

[0163] In the case that the number of RAW images included in the ZSL queue is greater than or equal to the preset number, step S902 is performed.

[0164] S902. The frame selection module determines the number of RAW images included in the frame selection queue.

[0165] In the case that the number of RAW images included in the frame selection queue is greater than or equal to the preset number, step S702 is performed.

[0166] In the case that the number of RAW images included in the frame selection queue is less than the preset number, step S903 is performed.

[0167] S903. The frame selection module acquires the shooting parameter data of the RAW image from the ZSL queue in a first order. The first order can be the order of the acquisition time from late to early. In step S903, the frame selection module acquires the RAW image that the frame selection module has not acquired in the process of performing steps S900-S909 this time.

[0168] Optionally, the ZSL queue includes a consumer queue and a producer queue. The frame selection module acquires the shooting parameter data of the RAW image from the consumer queue in a first order, and acquires the RAW image from the consumer queue in a second order. The second order is the order of the acquisition time from early to late. In this implementation, since the RAW image corresponding to the shooting parameter data in the producer queue has not been acquired by the camera, the shooting parameter data of the RAW image can be acquired from the producer queue in the order of the acquisition time from early to late. In this way, when it is confirmed that the RAW image is placed in the frame selection queue, the RAW image can be acquired by the camera more quickly and placed in the frame selection queue. In this way, the speed of generating a photo by the mobile phone can be further improved, and the time of see2review can be further reduced.

[0169] S904. The frame selection module determines whether the target RAW image is before the frame switching.

[0170] The target RAW image can be the RAW image corresponding to the shooting parameter data acquired by the frame selection module in step S903.

[0171] As a possible implementation, the frame selection module can determine whether the target RAW image is before the switching frame by the value of the flag bit B of the shooting parameter data of the previous frame RAW image of the target RAW image. When the value of the flag bit B of the shooting parameter data of the previous frame RAW image of the target RAW image is the first value, the target RAW image is before the switching frame. When the value of the flag bit B of the shooting parameter data of the previous frame RAW image of the target RAW image is the second value, the target RAW image is after the switching frame.

[0172] In the case that the target RAW image is before the switching frame, the above step S901 is performed.

[0173] In the case that the target RAW image is not before the switching frame, step S905 is performed.

[0174] S905. The frame selection module determines whether the target RAW image is a variable exposure frame.

[0175] As a possible implementation, the frame selection module can determine whether the target RAW image is a variable exposure frame by the value of the flag bit D of the shooting parameter data of the target RAW image. When the value of the flag bit D of the target RAW image is the first value or the second value, the target RAW image is a variable exposure frame. When the value of the flag bit D of the target RAW image is the third value, the target RAW image is not a variable exposure frame.

[0176] In the case that the target RAW image is a variable exposure frame, the above step S901 is performed.

[0177] In the case that the target RAW image is not a variable exposure frame, step S906 is performed.

[0178] S906. The frame selection module determines whether the target RAW image is an abnormal state.

[0179] As a possible implementation, the frame selection module can determine whether the target RAW image is an abnormal state by the value of the flag bit C of the shooting parameter data of the target RAW image. When the value of the flag bit C of the target RAW image is the first value, the target RAW image is an abnormal state. When the value of the flag bit C of the target RAW image is the second value, the target RAW image is not an abnormal state.

[0180] In the case that the target RAW image is an abnormal state, the above step S900 is performed.

[0181] In the case that the target RAW image is not an abnormal state, step S907 is performed.

[0182] S907. The frame selection module determines whether the difference between the collection time of the target RAW image and the shooting time is greater than the shooting time threshold.

[0183] The shooting moment can be a triggering moment of the shooting operation.

[0184] As a possible implementation, the frame selection module can determine whether the difference between the collection time of the target RAW image and the shooting moment is greater than the shooting time threshold through the timestamp included in the shooting parameter data of the target RAW image.

[0185] In the case where the frame selection module determines that the difference between the collection time of the target RAW image and the shooting moment is greater than the shooting time threshold, step S900 is performed.

[0186] In the case where the frame selection module determines that the difference between the collection time of the target RAW image and the shooting moment is less than or equal to the shooting time threshold, step S908 is performed.

[0187] S908. The frame selection module determines whether the target RAW image is a reference frame.

[0188] As a possible implementation, the frame selection module can determine whether the target RAW image is a reference frame through the flag bit A included in the shooting parameter data of the target RAW image. When the value of the flag bit A of the target RAW image is a first value, the target RAW image is a reference frame. When the value of the flag bit A of the target RAW image is a second value, the target RAW image is not a reference frame.

[0189] In the case where the target RAW image is a reference frame, step S900 is performed.

[0190] In the case where the target RAW image is not a reference frame, step S909 is performed.

[0191] S909. The frame selection module puts the target RAW image into the frame selection queue.

[0192] Optionally, in the case where the ZSL queue is traversed and the number of image frames included in the frame selection queue is less than a preset number, step S901 is performed. The ZSL queue being traversed can be understood as that the RAW image with the earliest collection time included in the ZSL queue is acquired by the frame selection module and subjected to the above determination.

[0193] In some embodiments, after the mobile phone performs step S702, the mobile phone obtains a photo. The mobile phone can further save the plurality of RAW images corresponding to the photo and the shooting parameter data corresponding to each RAW image to a target storage area. The target storage area is different from the storage area corresponding to the ZSL queue and the storage area corresponding to the frame selection queue. The target storage area can be a disk space of the mobile phone or a memory space of the mobile phone, and the embodiments of the present application do not limit this.

[0194] In the following, in the case of the idle period of the mobile phone, the mobile phone generates the enhanced photo using the image optimization algorithm on the stored multiple RAW images and the shooting parameter data corresponding to each RAW image.

[0195] In which, the idle period of the mobile phone can be obtained by the mobile phone analyzing the user's usage habits of the mobile phone. For example, the mobile phone analyzes that the user frequently uses the mobile phone from 8:00 to 18:00 every day, and the mobile phone can obtain that the idle period of the mobile phone is from 18:00 to 8:00 the next day.

[0196] Alternatively, the idle period of the mobile phone can also be preset, for example, 0:00-3:00 every day.

[0197] Alternatively, the idle period of the mobile phone can also be obtained by the mobile phone according to the processing occupation. For example, in the case that the processor occupation rate of the mobile phone is less than the occupation threshold, the mobile phone generates the enhanced photo using the image optimization algorithm on the stored multiple RAW images and the shooting parameter data corresponding to each RAW image.

[0198] Alternatively, the idle period of the mobile phone can also be that the camera application of the mobile phone stops running in the foreground. For example, in the case that the camera application changes to run in the background, the mobile phone generates the enhanced photo using the image optimization algorithm on the stored multiple RAW images and the shooting parameter data corresponding to each RAW image.

[0199] The technical solutions provided by the embodiments of the present application will be introduced below in combination with the use process of the user to the mobile phone.

[0200] Use process 1

[0201] In response to the operation of starting the camera application, the mobile phone displays the shooting preview interface. It can be understood that after the mobile phone displays the shooting preview interface, the mobile phone places the raw image captured by the camera into the first queue (such as the ZSL queue). In which, the first queue can also be understood as the first storage space, the first storage area, etc., and the above-mentioned first storage space, the first storage area are used to carry image data.

[0202] Next, at the first time point, in response to the first operation of the user, the mobile phone obtains the first photo according to the k frames of raw images in the first queue; the capture time of the k frames of raw images is before the first time point; k is an integer greater than or equal to 2. In which, the above-mentioned first shooting operation can be a single shooting operation, such as a click operation on the shooting button.

[0203] Afterwards, in response to a second shooting operation of the user, the mobile phone obtains a second photo according to the n frames of original images in the first queue; the collection time of the n frames of original images is before the second time point, and n is an integer greater than or equal to 2. The second time point is after the first time point. The second shooting operation can be a single shooting operation, such as a click operation on a shooting button.

[0204] Among them, the k frames of original images include m frames of original images in the n frames of original images; m is an integer greater than or equal to 1, and m is less than n and m is less than k.

[0205] That is, the mobile phone can obtain different photos according to part of the same original images. For example, the first photo is obtained according to the first original image and the second original image, and the second photo is obtained according to the second original image and the third original image. In this way, the same original image can be reused through different photos, and the mobile phone can obtain multiple photos through a relatively small number of original images, which can improve the frame selection efficiency of the mobile phone and save the storage space of the electronic device.

[0206] Optionally, after the mobile phone obtains the first photo, the mobile phone can also store the k frames of original images that constitute the first photo, for example, the mobile phone can store the first original image and the second original image. Afterwards, in the case that the mobile phone is idle, the mobile phone can obtain a first enhanced photo according to the first original image and the second original image. In the case that the mobile phone is idle, the processor occupancy of the mobile phone can be lower than an occupancy threshold, or the camera application can be closed, or the mobile phone can be idle in a time period, which can be obtained by the mobile phone by learning the usage habits of the user.

[0207] Among them, the mobile phone can use a first image optimization algorithm on the first original image and the second original image to obtain the first enhanced photo; and the mobile phone can use a second image optimization algorithm on the first original image and the second original image to obtain the first photo. The first image optimization algorithm is different from the second image optimization algorithm. For example, the mobile phone consumes more computing power when running the first image optimization algorithm than when running the second image optimization algorithm. It should be understood that during the process of taking a photo by the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original image. Therefore, in order to improve the quality of the picture, such as the aesthetic score of the picture, the mobile phone can use the image optimization algorithm that consumes more computing power to process the original image to obtain an enhanced photo in the idle time. In this way, the quality of the photo taken by the mobile phone can be improved.

[0208] Optionally, after the mobile phone obtains the first enhanced photo, the mobile phone can replace the first photo with the first enhanced photo, such as deleting the first photo and saving the first enhanced photo.

[0209] Process 2

[0210] In response to the operation of starting the camera application, the mobile phone displays a shooting preview interface. It can be understood that after the mobile phone displays the shooting preview interface, the mobile phone places the raw images captured by the camera into a first queue (for example, a ZSL queue). The first queue can also be understood as a first storage space, a first storage area, and the like. The above-mentioned first storage space and first storage area are used to carry image data.

[0211] Next, at a third time point, in response to a third shooting operation, a third photo is obtained according to x raw images, and a fourth photo is obtained according to y raw images. The third shooting operation can be multiple shooting operations, such as a long press operation on the shooting button.

[0212] Wherein, x is an integer greater than or equal to 2, y is an integer greater than or equal to 2. The x raw images include z raw images in the y raw images, z is an integer greater than or equal to 1, and z is less than x and y.

[0213] That is, the mobile phone can obtain different photos according to part of the same raw images. For example, the third photo is obtained according to the fourth raw image and the fifth raw image, and the fourth photo is obtained according to the fifth raw image and the sixth raw image. In this way, the same raw image can be reused through different photos, and the mobile phone can obtain multiple photos through a relatively small number of raw images, which can improve the frame selection efficiency of the mobile phone. Especially in the scene of continuous shooting of the mobile phone, the mobile phone needs to generate multiple photos in a short time. By reusing the same raw image through different photos, the mobile phone can obtain multiple photos through a relatively small number of raw images, which can improve the photo output efficiency of the mobile phone and save the storage space of the electronic device.

[0214] Optionally, after the mobile phone obtains the third photo, the mobile phone can also store the x raw images constituting the third photo, for example, the mobile phone can store the fourth raw image and the fifth raw image. Then, in the case that the mobile phone is idle, the mobile phone can obtain a third enhanced photo according to the fourth raw image and the fifth raw image. The above-mentioned case that the mobile phone is idle can be that the processor occupancy of the mobile phone is lower than an occupancy threshold, or can be that the camera application is closed, or can be a time period in which the mobile phone is idle, wherein the time period in which the mobile phone is idle can be obtained by the mobile phone learning the use habits of the user.

[0215] The third image optimization algorithm and the fourth image optimization algorithm are different. For example, the third image optimization algorithm consumes more computing power than the fourth image optimization algorithm. It should be understood that, in the process of photographing by the mobile phone, in order to shorten the see2review time, the mobile phone can use some lightweight algorithms to process the original image. Therefore, in order to improve the quality of the picture, such as the aesthetic score of the picture, the mobile phone can use the image optimization algorithm consuming more computing power to process the original image to obtain the enhanced picture during the idle time. In this way, the quality of the picture taken by the mobile phone can be improved.

[0216] It should be noted that the personal information used in the technical solutions of the present application is limited to information obtained with the individual consent of the person, including but not limited to informing and reminding the user to read the relevant user agreement (notification) before the user uses the function, and signing the agreement including authorization of relevant user information (authorization).

[0217] The algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered beyond the scope of the present application.

[0218] The present embodiment can divide the functional modules of the electronic device according to the above-mentioned method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware. It should be noted that the division of modules in the present embodiment is illustrative, and is only a logical functional division. Actual implementation can have another division method.

[0219] The present embodiment also provides an electronic device, as shown in the figure, which can include one or more processors 1801, a memory 1802, and a communication interface 1803. Figure 10 The memory 1802, the communication interface 1803, and the processor 1801 are coupled together. For example, the memory 1802, the communication interface 1803, and the processor 1801 can be coupled together through a bus 1804.

[0220] The memory 1802, the communication interface 1803, and the processor 1801 are coupled together. For example, the memory 1802, the communication interface 1803, and the processor 1801 can be coupled together through a bus 1804.

[0221] The communication interface 1803 is used for data transmission with other devices. The memory 1802 stores computer program code. The computer program code includes computer instructions, which, when executed by the processor 1801, cause the electronic device to perform the relevant method steps in the above-described method embodiments of this application.

[0222] The processor 1801 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0223] The bus 1804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The aforementioned bus 1804 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0224] This application also provides a chip system, such as... Figure 11 As shown, the chip system 2000 includes at least one processor 2001 and at least one interface circuit 2002. The processor 2001 and the interface circuit 2002 are interconnected via lines. For example, the interface circuit 2002 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 2002 can be used to send signals to other devices (e.g., the processor 2001). Exemplarily, the interface circuit 2002 can read instructions stored in the memory and send those instructions to the processor 2001. When the instructions are executed by the processor 2001, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, which are not specifically limited in this application embodiment.

[0225] The embodiment of the present application further provides a computer readable storage medium, which stores computer program codes, and when the processor executes the computer program codes, the electronic device executes the related method steps in the method embodiment.

[0226] The embodiment of the present application further provides a computer program product, which, when running on a computer, causes the computer to execute the related method steps in the method embodiment.

[0227] The electronic device, the computer readable storage medium or the computer program product provided by the present application are used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0228] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0229] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented by other means. For example, the device embodiment described above is only illustrative, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0230] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0231] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0232] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product in essence or in the part that contributes to the present application, or the whole or part of the technical solutions can be embodied in the form of a software product stored in a storage medium, including a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0233] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A shooting method, characterized in that, The method is applied to an electronic device, the electronic device including a camera, and the method includes: In response to launching the camera app, a preview interface is displayed; After the preview interface is displayed, the raw image captured by the camera is placed into the first queue; At a first time point, in response to a first shooting operation, a first photograph is obtained based on k original images in the first queue; the acquisition time of the k original images is before the first time point, and k is an integer greater than or equal to 2; At a second time point, in response to a second shooting operation, a second photograph is obtained based on n original images in the first queue; the second time point is later than the first time point, and the acquisition time of the n original images is before the second time point, where n is an integer greater than or equal to 2; Wherein, the k-frame original image includes m-frame original images from the n-frame original images; m is an integer greater than or equal to 1, and m is less than n, and m is less than k.

2. The method according to claim 1, characterized in that, The method further includes: At the third time, in response to the third shooting operation, a third photo is obtained based on the original image of frame x, and a fourth photo is obtained based on the original image of frame y; where x is an integer greater than or equal to 2, and y is an integer greater than or equal to 2. Wherein, the x-frame original image includes the z-frame original image in the y-frame original image, where z is an integer greater than or equal to 1, and z is less than x and z is less than y.

3. The method according to claim 1 or 2, characterized in that, The k-frame original images are the k original images that meet the frame selection criteria; the original images that meet the frame selection criteria include: original images that are not reference frames.

4. The method according to claim 3, characterized in that, The original images that meet the frame selection criteria also include: The original image whose acquisition time and the operation time of the first shooting operation are less than the shooting time threshold.

5. The method according to claim 3 or 4, characterized in that, The original images that meet the frame selection criteria also include one or more of the following: Not the original image of the variable exposure frame; The original image is not in an abnormal state; This is not the original image of the switched frame.

6. The method according to any one of claims 3-5, characterized in that, The step of obtaining the first photograph based on k original images in the first queue includes: Traverse the original images included in the first queue, and determine the original images that meet the frame selection criteria from the original images included in the first queue; The original image that meets the frame selection criteria is placed into the second queue; If the number of original images included in the second queue is greater than or equal to the k frames, the first photo is obtained based on the original images included in the second queue.

7. The method according to claim 6, characterized in that, The method further includes: If all the original images in the first queue have been traversed, and the number of original images in the second queue is less than the k frames, the camera is triggered to capture at least two frames of the target original images. A photograph is obtained based on at least two original images of the target.

8. The method according to claim 6, characterized in that, The method further includes: if the number of original images included in the first queue is less than the k frames, triggering the camera to capture at least two frames of target original images; A photograph is obtained based on at least two original images of the target.

9. The method according to any one of claims 6-8, characterized in that, The first queue includes a first sub-queue and a second sub-queue, and traversing the original images included in the first queue includes: The original images included in the first sub-queue are traversed in order of acquisition time from latest to earliest. The original images included in the second sub-queue are traversed in order of acquisition time from earliest to latest.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The first image optimization algorithm is used on the k original images to obtain the fifth photo. The first image optimization algorithm is different from the image optimization algorithm used on the first photo.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory; the processor is coupled to the memory; the memory is used to store computer program code; the computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-10.

13. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-10.

Images