Frame selection method and related device

Abstract

The embodiment of the invention provides a frame selection method and a related device, and relates to the technical field of terminals. The method comprises the following steps: in response to a pressing event of a photographing button, selecting M frames of images from N frames of images; wherein the N frames of images are images stored in a first queue before the pressing event occurs, and the images stored in the first queue are generated images; and selecting M frames of target images from the M frames of images and the Q frames of images under the condition that the Q frames of images to be acquired in the second queue are acquired and placed in the first queue, the second queue being used for setting frame identifiers for ungenerated images, and the M frames of target images being used for fusing and generating a photographed image. Therefore, the difference between the frame for fusing and generating the photographed image and the variable exposure frame can be reduced, and the fused image quality can be improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a frame selection method and related devices. Background Art

[0002] An electronic device can support a photographing function. When taking a photo, in order to improve the image quality, the electronic device can select multiple frames of images for image fusion processing, so as to obtain a better-quality image for display.

[0003] However, in some scenarios, the fused images obtained based on the existing frame selection methods have low quality, bringing a poor user experience. Summary of the Invention

[0004] Embodiments of this application provide a frame selection method and related devices, which are conducive to selecting image frames closer to the variable exposure frame, reducing the gap between the frames used for fusing to generate a photographed image and the variable exposure frame, and improving the quality of the fused image.

[0005] In a first aspect, an embodiment of this application proposes a frame selection method applied to an electronic device. The method includes: in response to a pressing event of a photographing button, selecting M frames of images from N frames of images; where the N frames of images are the images stored in a first queue before the pressing event occurs, and the images stored in the first queue are the already generated images; when Q frames of images to be collected in a second queue are all collected and put into the first queue, selecting M target frames of images from the M frames of images and the Q frames of images, where the second queue is used to set frame identifiers for ungenerated images, and the M target frames of images are used to fuse and generate a photographed image.

[0006] Optionally, the first queue may be referred to as a consumption queue, and the second queue may be included in a production queue, but this application does not make any limitations in this regard.

[0007] In an embodiment of this application, on the basis of selecting M frames of images from N frames of images, after the Q frames of images are collected, M target frames of images are selected from the M frames of images and the Q frames of images for fusing to generate a photographed image. The frame selection method provided by this application is conducive to the selected frames being closer to the variable exposure frame to be generated in the future based on the Q frames of images, reducing the gap between the selected frames and the variable exposure frame, and improving the quality of the photographed image.

[0008] In combination with the first aspect, in some implementation manners of the first aspect, before completing the selection of the M target frames of images, the method further includes: setting the Q frames of images to a non-recyclable state.

[0009] In connection with the first aspect, in certain implementations of the first aspect, setting the Q-frame image to a non-recyclable state includes: when the Q-frame image is placed in the first queue, modifying the reference count of the Q-frame image to a non-zero value.

[0010] It should be understood that an image being placed in the first queue means that the sensor has produced the image. In some examples, after the image frames in the second queue are actually produced, since they are no longer used for fusing the captured images, the reference counts of these frames are zero values, meaning that these frames can be recycled.

[0011] It should also be understood that the Q-frame images can be produced frame by frame, and the modification of the reference count for each frame in the Q-frame images can be carried out immediately after each frame is actually produced and placed in the first queue, without waiting to uniformly modify the reference counts after all the Q-frame images are produced.

[0012] In the embodiments of the present application, after the image frames in the second queue are actually produced, their reference counts are set to non-zero values, for example, they can be set to 1. In this way, their lifecycles can be extended, preventing these frames from being automatically recycled by the recycling mechanism set by the electronic device due to long-term non-use. After all the image frames in the second queue are produced, they can participate in frame selection.

[0013] In connection with the first aspect, in certain implementations of the first aspect, after completing the selection of the M-frame target images, the method further includes: setting the unselected images in the Q-frame images to a recyclable state.

[0014] In connection with the first aspect, in certain implementations of the first aspect, setting the unselected images in the Q-frame images to a recyclable state includes: modifying the reference counts of the unselected images in the Q-frame images to zero values.

[0015] In the embodiments of the present application, after completing the selection of the M-frame target images, the unselected images in the Q-frame images can be considered as images no longer needed in this capture process. Modifying their reference counts to zero values is beneficial for releasing these images and saving the memory of the electronic device.

[0016] In connection with the first aspect, in certain implementations of the first aspect, in response to a press event of the capture button, the method further includes: sending down target capture parameters, where the exposure parameter in the target capture parameters is different from the exposure parameter when acquiring the N-frame images and the Q-frame images; acquiring L-frame images obtained based on the target capture parameters, where the L-frame images and the M-frame target images are both used for fusing to generate the captured image; the Q-frame images are: the images being acquired but not generated between the first frame in the L-frame images and the last frame in the first queue when the press event occurs.

[0017] It should be understood that the sensor can collect images such as normal exposure image frames, long exposure image frames, and short exposure image frames. Among them, the normal exposure frames can be used as reference frames to provide a reference brightness; the long exposure image frames can provide information about dark areas, so that the details of the dark areas can also be presented in the photo; the short exposure frames can provide information about high-brightness areas and minimize the overexposed areas in the photo. The captured image obtained by fusing the normal exposure image frame, the long exposure image frame, and the short exposure image frame can present more image details, which is beneficial to improving the image quality.

[0018] Exemplarily, the N-frame image and the Q-frame image can be understood as normal exposure image frames, and the L-frame image obtained based on the target shooting parameters can be understood to include a long exposure image frame and / or a short exposure image frame, where L is a positive integer greater than or equal to 1. In the embodiments of the present application, the numbers of the long exposure image frame and the short exposure image frame in the L-frame image are not specifically limited.

[0019] Optionally, the Q-frame image can be all or part of the images between the first frame in the L-frame images and the last frame in the first queue when the pressing event occurs, and the images being captured but not yet generated. The present application does not limit this.

[0020] In combination with the first aspect, in some implementation manners of the first aspect, the frame identifiers in the first queue and the second queue are consecutive, and Q is equal to the difference between the frame identifier of the first frame in the L-frame images and the frame identifier of the last frame in the first queue when the pressing event occurs.

[0021] In a possible implementation manner, the frame identifiers may not be consecutive, but have a certain pattern. For example, they may be an arithmetic progression with a common difference not equal to 1, etc. The value of Q can be calculated based on a preset common difference and mathematical pattern. The present application does not limit this.

[0022] In combination with the first aspect, in some implementation manners of the first aspect, in response to a pressing event of the shooting button, the method further includes: sending a shooting instruction, and in response to the shooting instruction, fusing the L-frame images and the M-frame target images to obtain the captured image.

[0023] It should be understood that in response to a pressing event of the shooting button, when the electronic device detects 3A convergence based on the 3A algorithm, a shooting instruction is sent to generate a captured image based on the fusion of the L-frame images and the M-frame target images.

[0024] In combination with the first aspect, in some implementation manners of the first aspect, the electronic device includes a hardware abstraction layer, and the first queue and / or the second queue is a queue of the hardware abstraction layer.

[0025] Second aspect, the present application further provides a frame selection method, which is applied to an electronic device. The method includes: at a first moment, in response to a pressing event of a photographing button, the electronic device selects M1 frames of first images from A1 frame images, where the A1 frame images include C frame images generated before the pressing event occurs and D1 frame images being generated at the time when the pressing event occurs; generating a first photographed image by using the M1 frames of first images; at a second moment, in response to a pressing event of a photographing button, the electronic device selects M2 frames of second images from A2 frame images, where the A2 frame images include C frame images generated before the pressing event occurs and D2 frame images being generated at the time when the pressing event occurs; where M1 is different from M2, and D1 is different from D2; generating a second photographed image by using the M2 frames of second images.

[0026] In an embodiment of the present application, when the pressing event occurs, the electronic device does not immediately perform a frame selection operation, but waits until all the frames being generated at the time when the pressing event occurs are output, and then selects target image frames from these frames and the frames that have been generated at the time when the pressing event occurs for fusion to generate a photographed image. In this way, the frames being generated at the time when the pressing event occurs can also participate in the frame selection after being output, which is beneficial to selecting image frames closer to the variable exposure frame and is beneficial to improving the quality of the fused image.

[0027] Combined with the second aspect, in some implementation manners of the second aspect, the C frame images are the images stored in a first queue before the pressing event occurs. The images stored in the first queue are the images that have been generated. The D1 frame images are the images in a second queue, and the second queue is used to set frame identifiers for the images that have not been generated; before completing the selection of the M1 frames of first images, the method further includes: setting the D1 frame images to a non-recyclable state.

[0028] In a possible implementation manner, the first queue may be referred to as a consumption queue, and the second queue may be included in a production queue, but the present application does not make any limitation thereto.

[0029] Combined with the second aspect, in some implementation manners of the second aspect, the electronic device includes a hardware abstraction layer, and the first queue and / or the second queue are queues of the hardware abstraction layer.

[0030] Combined with the second aspect, in some implementation manners of the second aspect, the setting the D1 frame images to a non-recyclable state includes: when the D1 frame images are put into the first queue, modifying the reference count of the D1 frame images to a non-zero value.

[0031] In the embodiments of the present application, after the image frames in the second queue are actually produced, their reference counts are set to non-zero values, for example, they can be set to 1. In this way, their lifecycles can be extended, preventing these frames from being automatically recycled by the recycling mechanism set by the electronic device due to long-term non-utilization. After all the image frames in the second queue are produced, they can participate in frame selection.

[0032] In combination with the second aspect, in some implementation manners of the second aspect, after the first image selection of the M1 frame is completed, the method further includes: setting the unselected images in the D1 frame image to a recyclable state.

[0033] In combination with the second aspect, in some implementation manners of the second aspect, setting the unselected images in the D1 frame image to a recyclable state includes: modifying the reference counts of the unselected images in the D1 frame image to zero values.

[0034] In the embodiments of the present application, after the first image selection of the M1 frame is completed, the unselected images in the D1 frame image can be considered as the images no longer needed in this photographing process. Modifying their reference counts to zero values is beneficial to releasing this part of the images and saving the memory of the electronic device.

[0035] In combination with the second aspect, in some implementation manners of the second aspect, in response to a press event of the photographing button, the method further includes: sending down target photographing parameters, where the exposure parameter in the target photographing parameters is different from the exposure parameter when the A1 frame image is acquired; acquiring an L1 frame image obtained based on the target photographing parameters, where the L1 frame image and the first image of the M1 frame are both used for fusing to generate the first photographed image; the D1 frame image is: the images being acquired but not generated between the first frame in the L1 frame image and the last frame in the first queue when the press event occurs.

[0036] In the embodiments of the present application, the A1 frame image can be understood as a normal exposure image frame, and the L1 frame image obtained based on the target photographing parameters can be understood as including long exposure image frames and / or short exposure image frames, where L1 is a positive integer greater than or equal to 1. The present application embodiments do not make specific limitations on the number of long exposure image frames and short exposure image frames in the L1 frame image. The photographed image obtained by fusing the normal exposure image frame, long exposure image frames, and short exposure image frames can present more image details, which is beneficial to improving the image quality.

[0037] In combination with the second aspect, in some implementation manners of the second aspect, the frame identifiers in the first queue and the second queue are consecutive, and D1 is equal to the difference between the frame identifier of the first frame in the L1 frame image and the frame identifier of the last frame in the first queue when the press event occurs.

[0038] In another possible implementation, the frame identifiers in the first queue and the second queue may be discontinuous. For example, but with a certain pattern, such as an arithmetic progression with a common difference not equal to 1. The value of D1 can be calculated based on a preset common difference and mathematical rules, and this application does not limit this.

[0039] Combined with the second aspect, in some implementations of the second aspect, in response to a pressing event of the photographing button, the method further includes: sending a photographing instruction, and in response to the photographing instruction, fusing the L1 frame image and the M1 frame first image to obtain the first photographed image.

[0040] In a third aspect, an embodiment of the present application provides a frame selection device. The frame selection device may be an electronic device, or a chip or a chip system inside the electronic device. The frame selection device may include a processing unit. When the frame selection device is an electronic device, the processing unit may be a processor. The frame selection device may further include a storage unit, and the storage unit may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the electronic device to implement a frame selection method described in any one of the possible implementations in the first aspect or the second aspect. When the frame selection device is a chip or a chip system inside the electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to enable the electronic device to implement a frame selection method described in any one of the possible implementations in the first aspect or the second aspect. The storage unit may be a storage unit inside the chip (for example, registers, caches, etc.), or a storage unit outside the chip and inside the electronic device (for example, read-only memory, random access memory, etc.).

[0041] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory. The memory is used to store code instructions, and the processor is used to run the code instructions to execute the method described in any one of the possible implementations in the first aspect or the second aspect.

[0042] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium. A computer program or instructions are stored in the computer-readable storage medium. When the computer program or instructions are run on a computer, the computer is enabled to execute the method described in any one of the possible implementations in the first aspect or the second aspect.

[0043] In a sixth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program is run on a computer, the computer is enabled to execute the method described in any one of the possible implementations in the first aspect or the second aspect.

[0044] In a seventh aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected by a circuit. The at least one processor is configured to run a computer program or instruction to execute the method described in any possible implementation manner of the first aspect or the second aspect. Among them, the communication interface in the chip can be an input / output interface, a pin, a circuit, etc.

[0045] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, and instructions are stored in the at least one memory. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or it can be a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0046] It should be understood that the third aspect to the seventh aspect of the present application correspond to the technical solutions of the first aspect or the second aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar, and will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of an interface of an electronic device provided by an embodiment of the present application;

[0048] Figure 2 It is a schematic diagram of an image frame queue provided by an embodiment of the present application;

[0049] Figure 3 It is a schematic diagram of a software architecture of an electronic device provided by an embodiment of the present application;

[0050] Figure 4 It is a schematic flowchart of a frame selection method provided by an embodiment of the present application;

[0051] Figure 5 It is an example diagram of a frame selection method provided by an embodiment of the present application;

[0052] Figure 6 It is a schematic flowchart of another frame selection method provided by an embodiment of the present application;

[0053] Figure 7 It is a schematic flowchart of yet another frame selection method provided by an embodiment of the present application;

[0054] Figure 8 It is a schematic flowchart of still another frame selection method provided by an embodiment of the present application;

[0055] Figure 9 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0056] Figure 10A schematic structural diagram of a chip provided by an embodiment of the present application. Detailed implementation manners

[0057] To facilitate a clear description of the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0058] 1. 3A module: The automatic exposure / automatic white balance / automatic focus (AE / AWB / AF) module is also called the 3A module. The 3A module is the basic algorithm module of a camera. Each "A" represents a basic component of imaging. AE corresponds to the basic functional module of the camera device, AWB is the basic module of the color system, and AF controls the focus motor module.

[0059] 2. RAW image: It can be understood as an image collected by the camera device without being processed. The RAW image can also be understood as the original image.

[0060] 3. YUV image: It can be understood as an image processed by using a color encoding method. Among them, "Y" represents luminance, that is, the grayscale value, "U" represents chrominance, which is used to describe the color of the image, and "V" represents chroma, which is used to describe the saturation of the image.

[0061] 4. ISP: The image signal processor (ISP) can be used to process images. Among them, image processing can include image rasterization, black level compensation, lens correction, bad pixel correction, color difference, noise removal, automatic white balance, color correction, color control conversion, downsampling processing, image format conversion, etc. In some implementations, the processing of images by the ISP can also be referred to as ISP image processing. For example, ISP image processing can convert a RAW image into a YUV image.

[0062] 5. ZSL queue: The zero shutter lag (ZSL) queue can be used to store the images collected by the image sensor of an electronic device. Among them, the image sensor can also be understood as a camera. The ZSL queue can include a consumer list and a producer list.

[0063] Consumption queue: The consumption queue includes the generated images, which can be understood as the images that the camera has completed collecting. These generated images can have corresponding information related to image clarity or image quality, such as gyro information for sensing jitter, ois information for optical image stabilization, etc. Optionally, the first queue in the embodiments of the present application can be understood as the consumption queue.

[0064] Production queue: The production queue includes the images that are being generated or have not been generated yet. These images that are being generated or have not been generated yet can also be called future frame images. The future frame image can be understood as the image that the camera has not completed collecting or has not started collecting yet. That is to say, the future frame image is not an actually generated image, but only has a corresponding number. Optionally, the second queue in the embodiments of the present application can be understood as belonging to the production queue.

[0065] 5. Preview image and captured image

[0066] The preview image can be the data collected in real time by the camera of the electronic device and used for display in the preview screen. For example, when the electronic device receives an operation to open the camera application from the user, the electronic device can collect the preview image captured by the camera and display it in real time in the preview screen of the camera application.

[0067] The captured image can be the data obtained based on the capture button in the electronic device. For example, when the electronic device receives a trigger operation from the user for the capture button, the electronic device can obtain the captured image obtained based on the camera at the moment of capture.

[0068] 6. HDR mode: The high dynamic range (HDR) mode can also be called the dual-frame HDR mode. The HDR mode can synthesize multiple photos with different exposure levels, making the brightness range of the photos wider and presenting more details and colors. Taking pictures in HDR mode can obtain multiple frames of images with different exposure parameters. For example, long-frame and short-frame images can be obtained in one capture.

[0069] 7. Other terms

[0070] In the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. For example, the first chip and the second chip are only used to distinguish different chips and do not limit their order. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and terms such as "first" and "second" do not necessarily limit them to be different.

[0071] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0072] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c may represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c may be single or multiple.

[0073] 8. Electronic device

[0074] The electronic devices in the embodiments of this application may include handheld devices with a photographing function, vehicle-mounted devices, etc. For example, some electronic devices are: mobile phones, tablet computers, handheld computers, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication functions, computing devices, or other processing devices connected to a wireless modem, vehicle-mounted devices, wearable devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc. The embodiments of this application are not limited thereto.

[0075] By way of example and not limitation, in the embodiments of this application, the electronic device may also be a wearable device. A wearable device may also be referred to as a wearable intelligent device, which is a general term for devices developed by applying wearable technologies to the intelligent design of daily wear, such as glasses, gloves, watches, clothing, and shoes, etc. A wearable device is a portable device that is either directly worn on the body or integrated into the user's clothing or accessories. A wearable device is not only a hardware device, but also realizes powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable intelligent devices include those with complete functions and large sizes that can realize complete or partial functions without relying on a smart phone, such as smart watches or smart glasses, etc., and those that only focus on a certain type of application function and need to cooperate with other devices such as smart phones, such as various smart bracelets and smart jewelry for physical sign monitoring.

[0076] In addition, in the embodiments of the present application, the electronic device may also be a terminal device in an Internet of Things (IoT) system. The IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and thing-thing interconnection.

[0077] The electronic device in the embodiments of the present application may also be referred to as: terminal device, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile terminal, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, etc.

[0078] In the embodiments of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement service processing through processes. For example, Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system, etc. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software.

[0079] Figure 1 An exemplary schematic diagram of the camera application interface of the electronic device is shown. Figure 1 In a, the shooting interface 101 of the camera application is shown. The user can take a photo by clicking the shooting button 105 in the interface 101. Optionally, the user can also select the shooting mode. For example, the user can enter the interface 102 shown in b through the "More" 103 option in the interface 101, and select "HDR" in the interface 102 to take a photo in HDR mode. It should be understood that not all controls on the shooting interface are shown in the embodiments of the present application. The shooting interface can be designed with more or fewer controls, and the present application does not limit this. The user holds the electronic device and aims at the target object (such as Figure 1 the object 104 shown in a), and clicks the shooting button 103, which can trigger the mobile phone to take a photo and obtain a photo of the target object. Optionally, after clicking the shooting button, the user can view the taken photo by clicking the control 105. Figure 1 In a, the shooting interface 101 of the camera application is shown. The user can take a photo by clicking the shooting button 105 in the interface 101. Optionally, the user can also select the shooting mode. For example, the user can enter the interface 102 shown in b through the "More" 103 option in the interface 101, and select "HDR" in the interface 102 to take a photo in HDR mode. It should be understood that not all controls on the shooting interface are shown in the embodiments of the present application. The shooting interface can be designed with more or fewer controls, and the present application does not limit this. The user holds the electronic device and aims at the target object (such as

[0080] In some scenarios, adopting the HDR mode can generate images with a relatively high dynamic range, and can better present the image details in bright and dark light. After the user presses the capture button, in response to the pressing event of the capture button, the electronic device can select multiple frames of images for image fusion processing, so as to obtain better-quality images for display.

[0081] In some examples, the capture mode of the electronic device can also be referred to as the ultra-fast capture mode, which is not limited in this application.

[0082] Figure 2 Exemplarily shows the image frame output process of the electronic device. As Figure 2 shown, at the third moment, in response to the user's operation of opening the camera (for example, it can be the interface shown in a in the display Figure 1 ), the electronic device can start collecting image frames. Before the user presses the capture button, the image frames collected by the electronic device can be used to display the preview image.

[0083] When the user presses the capture button at the fourth moment, there are a frame queue that has been generated up to the fourth moment, a frame queue that is being generated at the fourth moment, and a frame queue to be generated after the fourth moment in the electronic device. Among them, the frame queue that has been generated up to the fourth moment can be called the consumption queue, and the frame queue that is being generated at the fourth moment refers to the frame queue that the electronic device is collecting but has not yet generated at the moment when the capture button is pressed. The image frames in the frame queue to be generated after the fourth moment need to be generated based on the capture parameters issued at the fourth moment when the capture button is pressed. It should be understood that the frame queue that is being generated at the fourth moment and the frame queue to be generated after the fourth moment can be called the production queue. There are no actually generated image frames in the production queue, only the corresponding frame numbers.

[0084] Exemplarily, the image frames can be identified by frame numbers. The frame numbers of the image frames in the consumption queue can be frame 1, frame 2, frame 3, frame 4, frame 5, frame 6; the frame numbers of the image frames in the production queue can be frame 7, frame 8, frame 9, frame 10, frame 11, frame 12, frame 13; among them, frames 7 to 11 can represent the frames that are being generated at the fourth moment, and frames 12 and 13 can represent the frames to be generated after the fourth moment.

[0085] In some implementations, when the electronic device needs multiple frames of images for image fusion processing, the electronic device can select multiple frames of images from the consumption queue and perform image fusion processing on the multiple frames of images to generate the capture image. However, in a scenario with variable exposure, for example, it can be understood as the above Figure 2Frames 12 and 13 generated based on the photographing parameters issued at the fourth moment when the photographing button is pressed have exposure parameters different from those of the previously obtained frames. Frames 12 and 13 can be referred to as variable exposure frames. In this case, if only frames are selected from the consumption queue, for example, frames 3, 4, 5, and 6 in the consumption queue are selected for fusion to generate a photographed image, these frames of images are separated from the image frames generated based on the photographing parameters issued at the moment when the photographing button is pressed by 5 frames or more. For example, in a motion scenario, the relative positions of the target object in each frame of image may be different, and there may be a large difference in the RAW image screen between the frames selected from the consumption queue and the variable exposure frames, which may lead to a poor fusion effect between the frames selected from the consumption queue and the variable exposure frames generated at the moment when the photographing button is pressed. For example, there may be a ghosting situation in the photographed image, affecting the user's photographing experience.

[0086] In view of this, the present application proposes a frame selection method and related device, so that the frames being generated by the electronic device at the moment when the photographing button is pressed can also participate in frame selection, which is beneficial for the electronic device to select image frames closer to the moment when the photographing button is pressed, beneficial for improving the quality of the photographed image generated by fusing the selected image frames and the image frames generated at the moment when the photographing button is pressed, and beneficial for improving the user experience of using the electronic device.

[0087] For the above technical concept, the present application proposes the following two solutions:

[0088] Solution 1: The electronic device performs two frame selection processes. That is, the electronic device responds to the pressing event of the photographing button, performs a first frame selection based on the frames that have been generated when the pressing event occurs, and after all the frames being generated when the pressing event occurs are generated, performs a second frame selection among these frames and the frames selected in the first frame selection, and fuses the result of the second frame selection with the variable exposure frames to generate a photographed image.

[0089] Solution 2: The electronic device responds to the pressing event of the photographing button, and after all the frames being generated when the pressing event occurs are generated, performs frame selection among these frames and the frames that have been generated when the pressing event occurs, and fuses the frame selection result with the variable exposure frames to generate a photographed image.

[0090] It should be understood that the Figure 2 number of frames shown above is only exemplary. The image frames are not limited to being identified by frame numbers, and can also be identified by other symbols or numbers. The present application does not make any limitations in this regard.

[0091] Next, before introducing the frame selection method provided by the embodiments of the present application, the software architecture of the electronic device will be described first.

[0092] Figure 3This is a schematic diagram of the software architecture of the electronic device provided by the embodiments of this application. The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. The layered architecture can adopt the Android system, the iOS system, or other operating systems, and the embodiments of this application do not limit this. Taking the Android system with a layered architecture as an example below, the software architecture of the electronic device provided by the embodiments of this application will be exemplarily described.

[0093] As Figure 3 shown, the layered architecture can divide the software system of the electronic device into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom, namely the application layer (applications), the application framework layer (application framework), the hardware abstraction layer (hardware abstraction layer, HAL), the kernel layer (kernel), and the hardware layer.

[0094] The application layer can include a series of application packages, and the application layer runs the applications by calling the application programming interfaces (APIs) provided by the application framework layer. As Figure 3 shown, the application packages can include applications such as the camera and the gallery.

[0095] It can be understood that Figure 3 the applications shown in

[0096] are only some applications. In fact, the application layer can also include other applications, which are not limited in this application. For example, the application layer can also include applications such as information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, etc., which are not limited in this application. Figure 3 The application framework layer provides APIs and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. As

[0097] shown, the application framework layer can include a camera access interface. Among them, the camera access interface can be used to provide an application programming interface and a programming framework for the camera application.

[0098] Optionally, the camera access interface can be an Android Interface Definition Language (AIDL) interface, which is not limited in this application.The HAL layer may include a camera hardware abstraction layer, a camera algorithm library, a frame management module, and a queue module. Among them, the camera hardware abstraction layer may provide virtual hardware for camera devices. The camera algorithm library may include the running code and data for implementing the image processing method provided in the embodiments of the present application, such as 3A algorithms, post-processing algorithms, and frame selection algorithms. Correspondingly, the corresponding algorithms can be run through the 3A algorithm module, the post-processing algorithm module, and the frame selection module. The frame management module may be used for managing the life cycle of image frames, such as incrementing the reference count of an image frame to extend its life cycle. The queue module may be used for storing the above-mentioned consumption queue and / or production queue.

[0099] In some examples, the camera algorithm library is further used to send digital signals to the digital signal processor driver in the driver layer, so that the digital signal processor driver can call the digital signal processor in the hardware layer to perform digital signal processing. The digital signal processor can return the processed digital signals to the camera algorithm library through the digital signal processor driver. The camera algorithm library is also used to send digital signals to the graphics signal processor driver in the driver layer, so that the graphics signal processor driver can call the graphics processor in the hardware layer to perform digital signal processing. The graphics processor can return the processed graphic data to the camera algorithm library through the graphics processor driver.

[0100] The kernel layer is the layer between hardware and software. As Figure 3 shown, the kernel layer may include one or more of the following: a camera device driver, a digital signal processor driver, an image processor driver, etc. Among them, the camera device driver is used to drive the sensor of the camera to collect images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0101] The hardware layer may include: sensors, such as image acquisition sensors, which can also be understood as cameras, image signal processors (ISPs), digital signal processors, and graphics processing units (GPUs), etc.

[0102] It should be understood that in some embodiments, layers that implement the same function may be referred to by other names, or a layer that can implement the functions of multiple layers may be regarded as one layer, or a layer that can implement the functions of multiple layers may be divided into multiple layers. The embodiments of the present application do not limit this.

[0103] Next, in combination with the modules in the above Figure 3 shown software architecture and Figure 2 the timing shown, the frame selection method in the embodiments of the present application will be described from the perspective of the interaction between modules. Figure 4 and Figure 5For the specific description of the above Scheme 1, Figure 6 For the specific description of the above Scheme 2.

[0104] Figure 4 Exemplarily shown is a schematic flowchart of a frame selection method 400 provided by an embodiment of the present application. The method 400 can be executed by an electronic device with a photographing function, and the software architecture of the electronic device can be as described above Figure 3 as shown, but the present application does not limit this.

[0105] The method 400 includes the following steps:

[0106] At a third moment, in response to an operation by the user to open the camera application, the electronic device executes S401 and subsequent steps.

[0107] S401. The camera application calls the camera access interface of the application framework layer, and through the camera access interface, calls the camera device driver in the kernel layer of the camera hardware abstraction layer to start the sensor through the camera device driver.

[0108] It should be understood that the sensor in the embodiment of the present application can be understood as an image sensor, or referred to as a camera.

[0109] Optionally, the operation by the user to open the camera application can be, for example, an operation of clicking on the camera application icon, or an operation of waking up the intelligent voice assistant and using a voice command to open the camera application. The present application does not limit this.

[0110] It should be understood that the camera access interface can convert instructions from the application layer into instructions recognizable by the hardware abstraction layer. The camera application can send instructions to start the camera to the camera device driver in the kernel layer by calling the camera hardware abstraction layer. The camera device driver can drive the corresponding camera sensor to collect images.

[0111] S402. The sensor collects images.

[0112] It should be understood that the process of the sensor collecting images can be understood as the process of the sensor pixel array being exposed. After the exposure of the sensor pixel array is completed, the sensor generates a frame of image. The image collected by the sensor without being processed by the image signal processor can be referred to as a RAW image.

[0113] In a possible implementation, the electronic device executes S403 to S406, transmits the RAW image collected by the sensor without being processed by the image signal processor to the image signal processor, and after the image signal processor processes the RAW image into a YUV image, it is transmitted back to the camera application through the camera device driver, the camera hardware abstraction layer, and the camera access interface for display on the preview interface of the camera application. Exemplarily, Figure 1The target object 104 shown in a in

[0114] S403. The sensor transmits the RAW image to the image signal processor.

[0115] S404. The image signal processor converts the RAW image into a YUV image.

[0116] S405. The image signal processor transmits the YUV image to the camera application.

[0117] S406. The camera application displays the preview image.

[0118] It should be understood that the preview image displayed in the camera application may be the YUV image processed by the image signal processor.

[0119] In another possible implementation, the electronic device executes S407 to transmit the RAW image collected by the sensor without being processed by the image signal processor to the queue module.

[0120] S407. The sensor transmits the RAW image to the queue module.

[0121] Optionally, the sensor transmits the RAW image to the queue module through the camera device driver.

[0122] It should be understood that the electronic device may execute the processes of S403 to S406 and S407 simultaneously, or may execute the processes of S403 to S406 first and then execute S407, or may execute S407 first and then execute S403 to S406, or the respective steps in S403 to S406 and S407 are interspersed and executed. This application does not make any limitations in this regard.

[0123] At the fourth moment, the user presses the capture button, and the electronic device executes S408 and subsequent steps.

[0124] S408. The camera application sends a press event.

[0125] It should be understood that at the moment when the press event occurs, there is still a Z-frame image being generated in the sensor. The press event includes the capture parameters. The camera application transmits the press event to the camera hardware abstraction layer through the camera access interface. The camera hardware abstraction layer calls the camera device driver to transmit the capture parameters to the sensor, so that the sensor collects a W-frame variable exposure image based on the capture parameters after the Z-frame image is collected and transmits the W-frame variable exposure image to the queue module. Exemplarily, the capture parameter is a variable exposure parameter, that is, the exposure parameter of the image frame generated based on the capture parameter is different from the exposure parameters of the frames that have been generated and are being generated at the time when the press event occurs.

[0126] In a possible scenario, the above S401 to S407 may correspond to the steps executed by the electronic device after the third moment above. At the moment when S408 occurs, the RAW images that have been put into the consumption queue may be Frame 1, Frame 2, Frame 3, Frame 4, Frame 5, and Frame 6 above. The frames being generated may be Frame 7, Frame 8, Frame 9, Frame 10, and Frame 11 above. The variable exposure frames that need to be generated in the future may be Frame 12 and Frame 13 above, but the present application does not limit this. Figure 2 After the third moment in Figure 2 , the frames being generated may be Frame 7, Frame 8, Frame 9, Frame 10, and Frame 11 above. The variable exposure frames that need to be generated in the future may be Frame 12 and Frame 13 above, but the present application does not limit this. Figure 2 In Figure 2 , the RAW images that have been put into the consumption queue may be Frame 1, Frame 2, Frame 3, Frame 4, Frame 5, and Frame 6 above. The frames being generated may be Frame 7, Frame 8, Frame 9, Frame 10, and Frame 11 above. The variable exposure frames that need to be generated in the future may be Frame 12 and Frame 13 above, but the present application does not limit this. Figure 2 In Figure 2 , the RAW images that have been put into the consumption queue may be Frame 1, Frame 2, Frame 3, Frame 4, Frame 5, and Frame 6 above. The frames being generated may be Frame 7, Frame 8, Frame 9, Frame 10, and Frame 11 above. The variable exposure frames that need to be generated in the future may be Frame 12 and Frame 13 above, but the present application does not limit this. Figure 2 In Figure 2 , the RAW images that have been put into the consumption queue may be Frame 1, Frame 2, Frame 3, Frame 4, Frame 5, and Frame 6 above. The frames being generated may be Frame 7, Frame 8, Frame 9, Frame 10, and Frame 11 above. The variable exposure frames that need to be generated in the future may be Frame 12 and Frame 13 above, but the present application does not limit this.

[0127] In some examples, the sensor may collect image frames such as normal exposure image frames, long exposure image frames, and short exposure image frames. Among them, the normal exposure frames are used as reference frames to provide a reference brightness. The long exposure image frames can provide information about dark areas, so that the details of the dark areas can also be presented in the photo. The short exposure frames can provide information about the highlight areas and minimize the overexposed areas in the photo. Optionally, in the embodiments of the present application, the frames that have been generated and the frames being generated when the press event occurs, that is, the normal exposure frames, can be called the normal frames of Ev 0, which are used to indicate that the exposure parameters of these frames are the same or similar. The image frames that need to be generated in the future based on the shooting parameters issued at the moment when the press event occurs may include long exposure image frames and / or short exposure image frames, that is, variable exposure frames. The present application does not limit this.

[0128] It should also be understood that in response to the press event, the camera application also sends the press event to the frame selection module in the camera algorithm library through the camera hardware abstraction layer, so that the frame selection module executes S409 below.

[0129] S409. The frame selection module obtains X frame images from the queue module.

[0130] It should be understood that the queue module is used to store the image frames in the consumption queue and set frame numbers for the image frames that have not been generated in the production queue. The X frame images obtained by the frame selection module from the queue module are the image frames in the consumption queue.

[0131] Optionally, the X frame images may include all or part of the images in the consumption queue when the press event occurs. The value of X may be determined by the frame selection algorithm running in the frame selection module. The present application does not limit this.

[0132] In a possible implementation manner, the X frame images may be the last X frame images in the consumption queue when the press event occurs.

[0133] S410. The frame selection module selects Y frame images from the X frame images.

[0134] In a possible implementation, the frame selection module may select the Y-frame image for fusion to generate a captured image based on information such as the sensed jitter gyro information and optical image stabilization ois information of the X-frame image, which can reflect the image quality. Optionally, the Y-frame image includes 1 reference frame and Y - 1 sub-optimal frames relative to the reference frame.

[0135] The frame management module can manage the consumption queue and production queue in the queue module. The frame management module executes S411 through interaction with the queue module.

[0136] S411: When there is an image frame generated in the Z-frame image that the frame management module is generating when detecting the occurrence of a press event, the reference count of this frame image is incremented by 1.

[0137] It should be understood that the frame management module can manage the frames in the consumption queue and production queue. In a possible implementation, the frame management module can obtain the frame number of the image frame being generated in the production queue when the press event occurs from the queue module, and when the image frame corresponding to this frame number is placed in the consumption queue, increment its reference count by 1 to maintain the life cycle of this part of the frames and prevent this part of the frames from being recycled due to a zero reference count after being produced. Optionally, the frame management module can set the reference count of this part of the frames to any non-zero value, and this application does not make a limitation on this.

[0138] It should also be understood that the sensor captures images frame by frame, and the frame management module can increment the reference count of each frame generated and placed in the consumption queue in the image frames being generated.

[0139] In a possible implementation, the frame numbers are consecutive. The frame management module can determine the frame numbers that need to perform life cycle management this time based on the difference between the frame number of the first frame generated at the moment when the press event occurs and the frame number of the last frame in the consumption queue at the moment when the press event occurs. Optionally, the frame numbers can follow other rules, and the frame management module can know the frame numbers of the image frames that need to perform life cycle management this time according to the rules followed by the frame numbers, and this application does not make a limitation on this.

[0140] In the case where the last frame in the frames being generated at the moment when the press event occurs is generated and placed in the consumption queue, the queue module executes S412.

[0141] S412: The queue module sends the Z-frame image to the frame selection module.

[0142] S413: The frame selection module selects Y-frame target images from the Y-frame image and the Z-frame image.

[0143] In a possible implementation, the Y-frame image includes 1 reference frame. The frame selection module sorts the Y-1 frame images in the Y-frame image except the reference frame and the Z-frame image based on the jitter amount parameter, selects the Y-1 preferred frames, and uses these Y-1 preferred frames and the reference frame as the Y-frame target image. Optionally, the parameter for frame selection can also be any other image quality parameter, which is not limited in this application.

[0144] It should be understood that the Y-frame image is selected from the frames that have been generated when the press event occurs, the Z-frame image is selected from the frames that are being generated when the press event occurs, and the reference frame for generating the fused image can be selected from the frames that have been generated when the press event occurs, which is beneficial to meeting the user's what-you-see-is-what-you-get requirement, and is also beneficial to making the frame number of the Y-frame target image closer to the variable exposure frame generated after the press event, which is beneficial to improving the quality of the multi-frame variable exposure photo fusion image.

[0145] Optionally, after the frame selection module executes S413, the frame management module can set the reference count of the frames in the Y-frame image and the Z-frame image that are not selected to 0, so as to release the frames and save the storage space of the electronic device.

[0146] S414. The frame selection module transmits the Y-frame target image to the frame management module.

[0147] It should be understood that the frame selection module also transmits the image quality information corresponding to the Y-frame target image to the frame management module, such as the sensed jitter gyro information, optical image stabilization ois information, etc., which is not limited in this application.

[0148] S415. The frame management module obtains the W-frame variable exposure image from the queuing module.

[0149] It should be understood that in S408, the camera application has sent the shooting parameters to the sensor for collecting the W-frame variable exposure image. When the camera is in the on state, the actions of the sensor corresponding to S402 occur continuously. The sensor collects images frame by frame and sends the collected image frames to the hardware abstraction layer and puts them into the consumption queue.

[0150] S416. The frame management module transmits the Y-frame target image and the W-frame variable exposure image to the post-processing algorithm module.

[0151] Optionally, the post-processing algorithm can be a multi-frame RAW domain algorithm, or any other algorithm that can implement multi-frame fusion, which is not limited in this application.

[0152] It should also be understood that after the above S408, the camera application also sends a lift event to the 3A algorithm module in the camera algorithm library through the camera hardware abstraction layer (for example, it can be understood as the response event made by the camera application after the user presses the capture button and then lifts the hand), so that the 3A algorithm module executes the 3A algorithm, and when 3A convergence is detected, the 3A algorithm module executes S417 to notify the camera application to issue a capture instruction.

[0153] S417. The 3A algorithm module notifies the camera application of 3A convergence.

[0154] S418. The camera application issues a capture instruction to the post-processing algorithm module.

[0155] S419. The post-processing algorithm module performs fusion processing on the Y-frame target image and the W-frame variable exposure image to obtain a captured image.

[0156] Optionally, the post-processing algorithm module can create a buffer for performing fusion processing on the Y-frame target image and the W-frame variable exposure image based on the capture instruction, perform fusion processing on the Y-frame target image and the W-frame variable exposure image to obtain a captured image. Optionally, the captured image can be in jpeg format, but this application does not make any limitation in this regard.

[0157] Optionally, the captured image can be stored in the gallery, or can be displayed in the thumbnail display area of the camera application in the form of a thumbnail (such as Figure 1 the 105 area shown in a of

[0158] Next, Figure 5 the above method 400 will be described in detail through a specific example. As Figure 5 shown, the frame queue at the time of a single frame selection by the electronic device (which can be understood as when the press event occurs) includes the already output frame queue, the frame queue being generated, and the future variable exposure frame queue. In the figure, the image frames are identified by boxes, and the frame numbers are identified by Arabic numerals.

[0159] Exemplarily, based on the already output frame queue at the time of a single frame selection, frames with frame numbers 0, 1, 2, and 3 are selected, and the frame with frame number 0 is the reference frame. Figure 5 The step corresponding to label ① in Figure 5 can be that the frame management module obtains the frames being generated by the sensor at the time of a single frame selection, such as Figure 5The frames with frame numbers 4, 5, 6, and 7 shown in [figure] are sent to the frame selection module for secondary frame selection together with the frames selected in the first frame selection. Exemplarily, the frame numbers of the frames selected in the secondary frame selection are 0, 2, 4, and 7. The steps corresponding to labels ③ and ④ may be that the frame selection module sends the frames with frame numbers 0, 2, 4, and 7 selected in the secondary frame selection and the variable exposure frames (such as the frames with frame numbers 8 and 9) during the first frame selection to the multi-frame RAW domain photographing algorithm through the frame management module to obtain a photographing image in jpeg format.

[0160] Figure 6 Exemplarily, a schematic flowchart of another frame selection method 600 provided by an embodiment of the present application is shown. This method 600 may be executed by an electronic device with a photographing function, and the software architecture of the electronic device may be as shown above Figure 3 but the present application does not limit this.

[0161] This method 600 includes the following steps:

[0162] S601 to S608 are similar to S401 to S408 above and will not be described in detail here.

[0163] It should be understood that the frame management module can manage the frames in the consumption queue and the production queue. The frame management module executes S609 through interaction with the queue module.

[0164] S609. When there is an image frame generated in the Z-frame image being generated by the frame management module when a press event is detected, the reference count of this frame image is incremented by 1.

[0165] In a possible implementation, the frame management module can obtain the frame number of the image frame being generated in the production queue when the press event occurs from the queue module, and when the image frame corresponding to this frame number is placed in the consumption queue, increment its reference count by 1 to maintain the life cycle of this part of the frames and prevent this part of the frames from being recycled due to a zero reference count after being produced.

[0166] It should be noted that the number of Z-frame images being generated when the press event occurs is not necessarily the same at different times of the press event, because the electronic device can calculate the number of frames used for fusing to generate the photographing image according to different press event occurrence scenarios, including normal frames and variable exposure frames in the ZSL queue. Since the number of frames in the consumption queue of the ZSL queue is fixed at the moment when the press event occurs, the number of frames being generated in the corresponding production queue can be increased or decreased to execute S611.

[0167] In one example, in a low dynamic range (LDR) scenario, such as in the non-HDR mode of a camera application or the so-called Binning mode, at the moment when a press event occurs, a sub-module in the 3A algorithm module, such as the AE module, can calculate based on parameters such as ambient brightness that 1 normal frame + 2 variable exposure frames are required to be combined to generate a captured image. Correspondingly, the number of frames being generated at that time can be 4 frames. The frame selection module can select 1 normal frame from the frames that have been generated at the moment when the press event occurs and these 4 frames for fusing to generate the captured image.

[0168] In another example, in a high dynamic range (HDR) scenario, such as Figure 1 in the HDR mode shown, at the moment when a press event occurs, the AE module can calculate based on parameters such as ambient brightness that 4 normal frames + 2 variable exposure frames are required to be combined to generate a captured image. Correspondingly, the number of frames being generated can be increased by 3 frames on the basis of the above 4 frames. The rule can be understood as follows: relative to the number of normal frames calculated by the AE module in the non-HDR mode for fusing to generate the captured image (such as 1 frame in the above example), on this basis, for each additional normal frame used for fusing to generate the captured image, the number of frames being generated at the moment when the press event occurs can also be increased by 1. In this way, in the high dynamic range scenario, the number of frames being generated can be more, and subsequently, there are more frames available for frame selection, which is beneficial for selecting frames with better image quality and improving the quality of the captured image.

[0169] In one possible implementation, an upper limit can be set for the number of Z-frame images being generated at the moment when a press event occurs to shorten the latency of capturing and outputting an image. Exemplarily, the maximum number of Z-frame images being generated at the moment when a press event occurs can be 8 frames, but this application does not make any limitations in this regard.

[0170] It should be understood that in the above examples, the numbers of normal frames and variable exposure frames are only exemplary and do not constitute a limitation to this application. This application does not make specific limitations on the number of frames in any case.

[0171] It should also be understood that the explanation of the number of Z-frame images being generated at the moment when a press event occurs is equally applicable to the number of Z-frame images being generated at the moment when a press event occurs described in the above method 400, and this application does not make any restrictions in this regard.

[0172] When the frame management module detects that all the Z-frame images being generated at the moment when a press event occurs have been output, through interaction with the queue module, it executes S610.

[0173] S610: The frame management module transmits X-frame images and Z-frame images to the frame selection module.

[0174] It should be understood that the X-frame image is the image in the consumption queue when the press event occurs. The X-frame image may include all or part of the images in the consumption queue when the press event occurs. The value of X may be determined by the frame selection algorithm running in the frame selection module, and the present application does not limit this.

[0175] S610. The frame selection module selects Y-frame target images from the X-frame images and the Z-frame images.

[0176] It should be understood that the Y-frame target images can be understood as the normal frames selected from the ZSL queue for fusing to generate the captured image. In the embodiments of the present application, the number of Y is also variable when different press events occur.

[0177] S611 to S616 are similar to S414 to S419 in the above method 400, and will not be elaborated here.

[0178] In the embodiments of the present application, when the press event occurs, the frame selection module does not immediately perform the frame selection operation. Instead, it waits until all the frames being generated when the press event occurs are output, and then selects the target image frames from these frames and the frames that have been generated at the press moment for fusing to generate the captured image. In this way, it is beneficial to select the image frames closer to the variable exposure frame, which is beneficial to improving the quality of the fused image. In addition, in the embodiments of the present application, the number of frames being generated when the press event occurs is not determined. In a low dynamic range scene, reducing the number of frames being generated when the press event occurs is beneficial to saving the power consumption of the electronic device. In a high dynamic range scene, increasing the number of frames being generated when the press event occurs is beneficial to providing more frames for the frame selection module to select, so as to select a sufficient number of frames with image quality meeting the requirements for fusing to generate the captured image, which is beneficial to improving the image quality.

[0179] The data processing method of the embodiments of the present application will be described in detail below through specific embodiments. The following embodiments can be combined with each other or implemented independently, and the same or similar concepts or processes may not be elaborated in some embodiments.

[0180] Figure 7 It is a schematic flowchart of a frame selection method 700 provided for the embodiments of the present application. This method 700 can be executed by an electronic device with a photographing function, and the software architecture of the electronic device can be as Figure 3 shown, but the present application does not make specific limitations on this.

[0181] This method 700 includes the following steps:

[0182] S701. In response to a press event on the photographing button, select M frames of images from N frames of images; where the N frames of images are the images stored in the first queue before the press event occurs, and the images stored in the first queue are the images that have been generated.

[0183] S702. When all the Q-frame images to be collected in the second queue are collected and placed in the first queue, select M target images from the M-frame images and the Q-frame images. The second queue is used to set frame identifiers for the ungenerated images, and the M target images are used to fuse and generate a captured image.

[0184] In a possible implementation, the first queue can be understood as the queue where the image frames generated when the press event described above occurs, and can be called the consumption queue. The second queue can be included in the production queue described above, but the present application does not make any limitations in this regard.

[0185] In some examples, the press event in this embodiment can be understood as Figure 2 the event of pressing the capture button at the fourth moment shown. The N-frame images can be understood as all or part of all the image frames captured by the camera from the time when the camera application is opened and the camera starts to work (for example Figure 2 the third moment shown) until the moment when the press event occurs. For example, it can be all or part of frames 1 to 6 above Figure 2 , but the present application does not make specific limitations on whether the frame numbers of the N-frame images are continuous or whether the N-frame images are the last N frames in the consumption queue when the press event occurs; the M-frame images can be understood as the frames selected during the first frame selection process of the electronic device. For example, they can be Figure 2 frames 3 to 6 shown; the Q-frame images can include, for example, all or part of frames 7 to 11 above Figure 2 . M, N, and Q are all positive integers greater than or equal to 1, and the values of M and N can be the same or different. The present application does not make any limitations in this regard.

[0186] In the embodiment of the present application, on the basis of selecting M-frame images from the N-frame images, after the Q-frame images are collected, select M target images from the M-frame images and the Q-frame images to fuse and generate a captured image. Since the press event can carry capture parameters when it occurs, the frames generated based on these capture parameters are also used to fuse and generate a captured image. The frame selection method provided by the present application is conducive to the selected frames being closer to the frames generated based on these capture parameters, which is conducive to improving the quality of the captured image. Especially in a motion scene, when the frames generated based on these capture parameters are variable exposure frames, the captured image can present better texture and better quality.

[0187] As an optional embodiment, the electronic device includes a hardware abstraction layer, and the first queue and / or the second queue are queues of the hardware abstraction layer. For example, they can be in the queue module of the hardware abstraction layer.

[0188] In a possible implementation, the hardware abstraction layer of the electronic device further includes a frame selection module and a frame management module. The frame selection module can select frames for fusing to generate a captured image, and the frame management module can maintain the lifecycle of the image frames in the queue module.

[0189] As an optional embodiment, in response to a pressing event of the capture button, the method further includes: sending down target capture parameters, where the exposure parameter in the target capture parameters is different from the exposure parameter when acquiring N frames of images and Q frames of images; acquiring L frames of images obtained based on the target capture parameters, and both the L frames of images and M frames of target images are used for fusing to generate a captured image; the Q frames of images are: the images that are being acquired but not generated between the first frame in the L frames of images and the last frame in the first queue when the pressing event occurs.

[0190] It should be understood that the sensor can acquire image frames such as normal exposure image frames, long exposure image frames, short exposure image frames, etc. Among them, the normal exposure frames can be used as reference frames to provide a reference brightness; the long exposure image frames can provide information about dark areas, so that the details of the dark areas can also be presented in the photo; the short exposure frames can provide information about highlight areas to minimize the overexposed areas in the photo. The captured image fused from the normal exposure image frames, long exposure image frames, and short exposure image frames can present more image details, which is beneficial to improving the image quality. Optionally, the electronic device can implement the fusion and image output through the post-processing algorithm module in the hardware abstraction layer.

[0191] Exemplarily, the N frames of images and the Q frames of images can be understood as normal exposure image frames, and the L frames of images obtained based on the target capture parameters can be understood as including long exposure image frames and / or short exposure image frames. L is a positive integer greater than or equal to 1, and the embodiments of the present application do not make specific limitations on the number of long exposure image frames and short exposure image frames in the L frames of images.

[0192] Optionally, the Q frames of images can be all or part of the images that are being acquired but not generated between the first frame in the L frames of images and the last frame in the first queue when the pressing event occurs. The present application does not make any limitations on this.

[0193] As an optional embodiment, before completing the selection of the M frames of target images, the method further includes: setting the Q frames of images to a non-recyclable state.

[0194] In a possible implementation, the way to set the Q frames of images to a non-recyclable state can be: when the Q frames of images are put into the first queue, modify the reference count of the Q frames of images to a non-zero value.

[0195] It should be understood that putting an image into the first queue means that the sensor has produced the image. In some examples, after the image frames in the second queue are actually produced, since they are no longer used for fusing the captured images, the reference count of these frames is zero, which means that these frames can be recycled.

[0196] It should also be understood that the Q-frame images can be produced frame by frame, and the modification of the reference count for each frame in the Q-frame images can be performed immediately after each frame is actually produced and put into the first queue, without waiting to perform it uniformly after all the Q-frame images are produced.

[0197] In the embodiments of the present application, after the image frames in the second queue are actually produced, their reference counts are set to non-zero values, for example, they can be set to 1. In this way, their lifecycles can be extended, preventing these frames from being automatically recycled by the recycling mechanism set by the electronic device due to long-term non-use, and waiting for all the image frames in the second queue to be produced before they can participate in frame selection.

[0198] As an optional embodiment, after completing the selection of the M-frame target images, the method further includes: setting the unselected images in the Q-frame images to a recyclable state.

[0199] In a possible implementation manner, the way to set the unselected images in the Q-frame images to a recyclable state can be: modifying the reference count of the unselected images in the Q-frame images to zero.

[0200] In the embodiments of the present application, after completing the selection of the M-frame target images, the unselected images in the Q-frame images can be considered as the images no longer needed in this capture process. Modifying their reference counts to zero is beneficial to releasing this part of the images and saving the memory of the electronic device.

[0201] In some implementations, the electronic device can determine the number of Q-frame images through the pattern of frame identifiers to maintain the lifecycle of the Q-frame images.

[0202] In a possible implementation manner, the frame identifiers in the first queue and the second queue are consecutive, and Q is equal to the difference between the frame identifier of the first frame in the L-frame images and the frame identifier of the last frame in the first queue when the press event occurs.

[0203] In another possible implementation manner, the frame identifiers can be non-consecutive but have a certain pattern, for example, they can be an arithmetic sequence with a common difference not equal to 1, etc. The value of Q can be calculated based on the preset common difference and mathematical rules, and the present application does not limit this.

[0204] As an optional embodiment, in response to the press event of the capture button, the method further includes: sending a capture instruction, and in response to the capture instruction, fusing the L-frame images and the M-frame target images to obtain a captured image.

[0205] It should be understood that in response to a pressing event of the photographing button, when the electronic device detects 3A convergence based on the 3A algorithm, a photographing instruction is sent to generate a photographed image by fusing both the L-frame image and the M-frame target image.

[0206] Figure 8 FIG. 800 is a schematic flowchart of a frame selection method provided by an embodiment of the present application. The method 800 can be executed by an electronic device with a photographing function, and the software architecture of the electronic device can be as Figure 3 shown therein, but the present application does not make specific limitations thereto.

[0207] The method 800 includes the following steps:

[0208] S801. At a first moment, in response to a pressing event of the photographing button, the electronic device selects M1 frames of first images from the A1-frame image, and the A1-frame image includes C-frame images generated before the pressing event occurs and D1-frame images being generated when the pressing event occurs.

[0209] S802. Generate a first photographed image by using the M1 frames of first images.

[0210] S803. At a second moment, in response to a pressing event of the photographing button, the electronic device selects M2 frames of second images from the A2-frame image, and the A2-frame image includes C-frame images generated before the pressing event occurs and D2-frame images being generated when the pressing event occurs; wherein, M1 is different from M2, and D1 is different from D2.

[0211] S804. Generate a second photographed image by using the M2 frames of second images.

[0212] It should be understood that at different moments, the number of image frames being generated when the pressing event occurs is not fixed, because the electronic device can calculate the number of frames used for fusing to generate a photographed image according to different pressing event occurrence scenarios, and correspondingly adjust the number of image frames being generated for different pressing events. In this way, the electronic device can, in a low dynamic range scene, save the power consumption of the electronic device by reducing the number of frames being generated when the pressing event occurs, and in a high dynamic range scene, increase the number of frames being generated when the pressing event occurs to provide more frames for the frame selection module to select enough frames with image quality meeting the requirements for fusing to generate a photographed image, thereby improving the image quality of the output image.

[0213] In the embodiments of the present application, when a press event occurs, the electronic device does not immediately perform a frame selection operation. Instead, after all the frames being generated at the time of the press event are output, a target image frame is selected from these frames and the frames that have been generated at the time of the press moment for fusing to generate a captured image. In this way, the frames being generated at the time of the press event can also participate in frame selection after being output, which is beneficial to selecting an image frame closer to the variable exposure frame and improving the quality of the fused image.

[0214] In a possible implementation manner, the first moment or the second moment may be the fourth moment described above, but the present application does not limit this. Figure 2 The present application does not limit this.

[0215] In a possible implementation manner, the electronic device includes a frame selection module and a post-processing algorithm module as shown in Figure 3 The above S801 and S803 may be executed by the frame selection module, and S802 and S804 may be executed by the post-processing algorithm module.

[0216] As an optional embodiment, the C-frame image is an image stored in the first queue before the press event occurs. The images stored in the first queue are already generated images. The D1-frame image is an image in the second queue, and the second queue is used to set frame identifiers for ungenerated images. Before completing the selection of the first M1-frame image, the method further includes: setting the D1-frame image to a non-recyclable state.

[0217] In a possible implementation manner, the first queue can be understood as the queue where the image frames that have been generated at the time of the press event described above are located, and can be called the consumption queue. The second queue may be included in the production queue described above, but the present application does not limit this.

[0218] In a possible implementation manner, the electronic device includes a hardware abstraction layer, and the first queue and / or the second queue are queues of the hardware abstraction layer.

[0219] In a possible implementation manner, setting the D1-frame image to a non-recyclable state includes: when the D1-frame image is put into the first queue, modifying the reference count of the D1-frame image to a non-zero value.

[0220] It should be understood that putting an image into the first queue means that the sensor has output the image. In some examples, after the image frames in the second queue are actually output, since they are no longer used for fusing the captured image, the reference count of these frames is zero, which means that these frames can be recycled.

[0221] It should also be understood that the D1 frame images can be produced frame by frame, and the modification of the reference count of each frame image in the D1 frame images can be performed immediately after each frame image is actually produced and placed in the first queue, without waiting for all the D1 frame images to be produced and then unifiedly performed.

[0222] In a possible implementation, the electronic device includes a frame management module and a queue module as shown in Figure 3 The queue module can be used to store the first queue and maintain the second queue. The frame management module can manage the life cycle of the image frames in the queue module, that is, the frame management module can modify the reference count of the image frames.

[0223] In the embodiments of the present application, after the image frames in the second queue are actually produced, their reference counts are set to non-zero values, for example, they can be set to 1. In this way, the life cycle can be extended, preventing these frames from being automatically recycled by the recycling mechanism set by the electronic device due to long-term non-use, and waiting for all the image frames in the second queue to be produced before they can participate in frame selection.

[0224] As an optional embodiment, after the first image selection of the M1 frame is completed, the method further includes: setting the unselected images in the D1 frame images to a recyclable state.

[0225] In a possible implementation, setting the unselected images in the D1 frame images to a recyclable state includes: modifying the reference count of the unselected images in the D1 frame images to zero.

[0226] In the embodiments of the present application, after the first image selection of the M1 frame is completed, the unselected images in the D1 frame images can be considered as the images no longer needed in this photographing process. Modifying their reference counts to zero is beneficial to releasing these images and saving the memory of the electronic device.

[0227] As an optional embodiment, in response to the pressing event of the photographing button, the method further includes: sending down target photographing parameters, where the exposure parameter in the target photographing parameters is different from the exposure parameter when obtaining the A1 frame image; obtaining the L1 frame image based on the target photographing parameters, and both the L1 frame image and the first image of the M1 frame are used to fuse and generate the first photographed image; the D1 frame image is: the images being collected but not generated between the first frame in the L1 frame image and the last frame in the first queue when the pressing event occurs.

[0228] In the embodiments of the present application, the A1-frame image can be understood as a normally exposed image frame, and the L1-frame image obtained based on the target photographing parameters can be understood as including a long-exposure image frame and / or a short-exposure image frame, where L1 is a positive integer greater than or equal to 1. In the embodiments of the present application, there are no specific limitations on the number of long-exposure image frames and short-exposure image frames in the L1-frame image. The photographed image obtained by fusing the normally exposed image frame, the long-exposure image frame, and the short-exposure image frame can present more image details, which is beneficial to improving the image quality.

[0229] As an optional embodiment, in response to the pressing event of the photographing button, the method further includes: sending a photographing instruction, and in response to the photographing instruction, fusing the L1-frame image with the M1-frame first image to obtain a first photographed image.

[0230] In some implementations, the electronic device can determine the number of image frames being generated when the pressing event occurs through the rule of frame identifiers, so as to maintain the life cycle of this part of the image frames.

[0231] In a possible implementation, the frame identifiers in the first queue and the second queue are consecutive, and D1 is equal to the difference between the frame identifier of the first frame in the L1-frame image and the frame identifier of the last frame in the first queue when the pressing event occurs.

[0232] In another possible implementation, the frame identifiers in the first queue and the second queue may be non-consecutive, for example, but with a certain rule, such as an arithmetic sequence with a common difference not equal to 1, etc. The value of D1 can be calculated based on the preset common difference and mathematical rules, and the present application does not limit this.

[0233] It should also be understood that the magnitudes of the serial numbers of the above methods do not mean the order of execution. The order of execution of each method should be determined by its function and internal logic.

[0234] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

[0235] The frame selection method of the embodiments of the present application has been described above. Next, the device for executing the above method provided by the embodiments of the present application will be described. Those skilled in the art can understand that the method and the device can be combined and referenced with each other. The relevant device provided by the embodiments of the present application can execute the steps in the above-listed methods.

[0236] The data processing method provided by the embodiments of the present application can be applied to an electronic device with a photographing function. The hardware structure of the electronic device can be as follows Figure 9 as shown below.

[0237] Exemplarily, Figure 9 FIG. shows a schematic diagram of the hardware structure of an electronic device provided by the embodiments of the present application.

[0238] As Figure 9 shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194, etc.

[0239] Optionally, the above-mentioned sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0240] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. It should be understood that the processor 110 may be used to execute the frame selection method provided by the embodiments of the present application.

[0241] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions, for example, the program codes for implementing the above embodiments. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device (such as audio data, a phone book, etc.). In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor.

[0242] The camera 193 is used to capture static images or videos. In some embodiments, the electronic device can include one or N cameras 193, where N is a positive integer greater than 1. In the embodiments of the present application, the camera can be used to collect image frames to implement the photographing function.

[0243] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device can include one or N display screens 194, where N is a positive integer greater than 1. The electronic device realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.

[0244] The electronic device realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information. The electronic device can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0245] It should be understood that in order to implement the functions described in the above embodiments, an electronic device may include a corresponding hardware structure and / or software module for performing each function. Those skilled in the art should easily realize that, in combination with the method steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0246] The embodiments of the present application can divide the device for implementing the method into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0247] Figure 10 It is a schematic structural diagram of a chip provided by an embodiment of the present application. The chip 1000 includes one or more than two (including two) processors 1001, a communication line 1002, a communication interface 1003, and a memory 1004.

[0248] In some embodiments, the memory 1004 stores the following elements: executable modules or data structures, or subsets thereof, or extended sets thereof.

[0249] The methods described in the above embodiments of the present application can be applied to the processor 1001 or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 1001 or by instructions in software form. The above-mentioned processor 1001 may be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate, transistor logic devices, or discrete hardware components. The processor 1001 can implement or execute the various processing-related methods, steps, and logic block diagrams disclosed in the embodiments of the present application.

[0250] The steps of the method disclosed in the embodiments of the present application can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the art such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read-only memory (EEPROM). This storage medium is located in the memory 1004, and the processor 1001 reads the information in the memory 1004 and combines its hardware to complete the steps of the above method.

[0251] The processor 1001, the memory 1004, and the communication interface 1003 can communicate through the communication line 1002.

[0252] In the above embodiments, the instructions stored in the memory for the processor to execute can be implemented in the form of a computer program product. Among them, the computer program product can be pre-written in the memory, or can be downloaded and installed in the memory in the form of software.

[0253] In the embodiments of the present application, the above chip 1000 can also be a chip system, for example: a system on chip (SoC), and the present application does not make any limitation in this regard.

[0254] The embodiments of the present application provide an electronic device, including: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0255] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method can be implemented. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. The computer-readable medium can include a computer storage medium and a communication medium, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0256] In one possible implementation, the computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM), or other optical disc storage, magnetic disk storage, or any other medium targeted to carry or store the required program code in the form of instructions or data structures and accessible by a computer. Moreover, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. As used herein, disk and optical disc include optical disc, laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while optical discs utilize lasers to optically reproduce data. Combinations of the above should also be included within the scope of computer-readable media.

[0257] An embodiment of the present application provides a computer program product. The computer program product includes a computer program that, when run, causes a computer to execute the above method.

[0258] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to generate a machine such that the instructions executed by the processing unit of the computer or other programmable data processing device generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0259] The above specific implementation manners further elaborate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific implementation manners of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included within the protection scope of the present invention.

Claims

1. A frame selection method, characterized in that, Applied to an electronic device, the method includes: In response to a pressing event of a capture button, select M frames of images from N frames of images; wherein, the N frames of images are images stored in a first queue before the pressing event occurs, and the images stored in the first queue are already generated images; In the case where Q frames of images to be captured in a second queue are all captured and placed in the first queue, select M frames of target images from the M frames of images and the Q frames of images, the second queue is used to set frame identifiers for ungenerated images, and the M frames of target images are used to generate a captured image by fusion.

2. The method according to claim 1, wherein Before completing the selection of the M frames of target images, the method further includes: Set the Q frames of images to a non-recyclable state.

3. The method according to claim 2, wherein The setting the Q frames of images to a non-recyclable state includes: When the Q frames of images are placed in the first queue, modify the reference count of the Q frames of images to a non-zero value.

4. The method according to claim 2 or 3, characterized in that, After completing the selection of the M frames of target images, the method further includes: Set the unselected images in the Q frames of images to a recyclable state.

5. The method according to claim 4, characterized in that, The setting the unselected images in the Q frames of images to a recyclable state includes: Modify the reference count of the unselected images in the Q frames of images to zero.

6. The method according to any one of claims 1 to 5, characterized in that, In response to a pressing event of a capture button, the method further includes: Send down target capture parameters, where the exposure parameter in the target capture parameters is different from the exposure parameter when obtaining the N frames of images and the Q frames of images; Obtain L frames of images based on the target capture parameters, the L frames of images and the M frames of target images are both used to generate the captured image by fusion; the Q frames of images are: the images that are being captured but not generated between the first frame in the L frames of images and the last frame in the first queue when the pressing event occurs.

7. The method according to claim 6, wherein The frame identifiers in the first queue and the second queue are consecutive, and Q is equal to the difference between the frame identifier of the first frame in the L frames of images and the frame identifier of the last frame in the first queue when the pressing event occurs.

8. The method according to claim 6 or 7, characterized in that In response to a pressing event of a capture button, the method further includes: Send down a capture instruction, and in response to the capture instruction, fuse the L frames of images and the M frames of target images to obtain the captured image.

9. The method according to any one of claims 1 to 8, characterized in that, The electronic device includes a hardware abstraction layer, and the first queue and / or the second queue are queues of the hardware abstraction layer.

10. A frame selection method, characterized in that, Applied to an electronic device, the method includes: At a first moment, in response to a pressing event of a capture button, the electronic device selects M1 frames of first images from A1 frames of images, where the A1 frames of images include C frames of images that have been generated before the pressing event occurs and D1 frames of images that are being generated when the pressing event occurs; Generate a first captured image using the M1 frames of first images; At a second moment, in response to a pressing event of a capture button, the electronic device selects M2 frames of second images from A2 frames of images, where the A2 frames of images include C frames of images that have been generated before the pressing event occurs and D2 frames of images that are being generated when the pressing event occurs; wherein, M1 is different from M2, and D1 is different from D2; Generate a second captured image by using the second image of the M2 frame.

11. The method according to claim 10, wherein The C-frame image is the image stored in the first queue before the pressing event occurs. The images stored in the first queue are already generated images. The D1-frame image is the image in the second queue, and the second queue is used to set frame identifiers for ungenerated images. Before completing the selection of the first image of the M1 frame, the method further includes: Set the D1-frame image to a non-recyclable state.

12. The method according to claim 11, wherein The setting the D1-frame image to a non-recyclable state includes: When the D1-frame image is put into the first queue, modify the reference count of the D1-frame image to a non-zero value.

13. The method according to claim 11 or 12, characterized in that, After completing the selection of the first image of the M1 frame, the method further includes: Set the unselected images in the D1-frame image to a recyclable state.

14. The method according to claim 13, wherein The setting the unselected images in the D1-frame image to a recyclable state includes: Modify the reference count of the unselected images in the D1-frame image to zero.

15. The method according to any one of claims 11 to 14, characterized in that, In response to a pressing event of the capture button, the method further includes: Send down target capture parameters, where the exposure parameter in the target capture parameters is different from the exposure parameter when obtaining the A1-frame image; Obtain an L1-frame image based on the target capture parameters. The L1-frame image and the first image of the M1 frame are both used to fuse and generate the first captured image. The D1-frame image is: the images that are being captured but not generated between the first frame in the L1-frame image and the last frame in the first queue when the pressing event occurs.

16. The method according to claim 15, wherein The frame identifiers in the first queue and the second queue are consecutive, and D1 is equal to the difference between the frame identifier of the first frame in the L1-frame image and the frame identifier of the last frame in the first queue when the pressing event occurs.

17. An electronic device, characterized in that, Includes: A processor and a memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 9, or executes the method according to any one of claims 10 to 16.

18. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 9, or implements the method according to any one of claims 10 to 16.

19. A chip system, characterized in that, Includes at least one processor and a communication interface. The communication interface and the at least one processor are interconnected by a line. The at least one processor is used to run a computer program or instruction to execute the method according to any one of claims 1 to 9, or execute the method according to any one of claims 10 to 16.

20. A computer program product, characterized in that, Includes a computer program. When the computer program is run, it causes the computer to execute the method according to any one of claims 1 to 9, or execute the method according to any one of claims 10 to 16.

Images