Shooting method and device

Abstract

The invention discloses a shooting method and device, and belongs to the technical field of image processing, and the method comprises the steps: collecting a first preview image set based on exposure information corresponding to a first shooting scene, the exposure information comprising a first long exposure duration and a first short exposure duration, the first preview image set comprises a long exposure image frame corresponding to the exposure information at a first frame rate and a short exposure image frame corresponding to the long exposure image frame at the first frame rate; the first image is displayed based on image frames in the first preview image set, a second preview image set is cached, the second preview image set comprises long exposure image frames under a second frame rate and short exposure image frames corresponding to the long exposure image frames under the first frame rate, and the second frame rate is smaller than the first frame rate; and generating a second image based on the second preview image set under the condition that a shooting instruction is received.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a shooting method and apparatus. Background Technology

[0002] Currently, with the rapid development of electronic devices, photography has become a very important application scenario, and shooting moving objects also occupies a very important proportion of many photography scenarios.

[0003] In related technologies, before shooting a moving object, the user can trigger the electronic device to enter a high frame rate shooting mode, such as 60 frames or 120 frames, so that the electronic device can capture a clear image containing the moving object in the high frame rate shooting mode.

[0004] However, in the above method, since the higher the frame rate, the more image frames the electronic device needs to process, the more system resources the electronic device needs, which in turn leads to higher power consumption when the electronic device captures images in high frame rate mode. Summary of the Invention

[0005] The purpose of this application is to provide a shooting method and apparatus that can reduce the power consumption of electronic devices when shooting images in high frame rate mode.

[0006] In a first aspect, embodiments of this application provide a shooting method, which includes: acquiring a first preview image set based on exposure information corresponding to a first shooting scene, wherein the exposure information includes a first long exposure duration and a first short exposure duration, and the first preview image set includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate; displaying a first image based on the image frames in the first preview image set, and caching a second preview image set, wherein the second preview image set includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, wherein the second frame rate is less than the first frame rate; and generating a second image based on the second preview image set upon receiving a shooting instruction.

[0007] Secondly, embodiments of this application provide a shooting device, which includes an acquisition module and a processing module. The acquisition module is used to acquire a first set of preview images based on exposure information corresponding to a first shooting scene when the electronic device is in a first shooting scene. The exposure information includes a first long exposure duration and a first short exposure duration. The first set of preview images includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. The processing module is used to display a first image based on the image frames in the first set of preview images and to cache a second set of preview images. The second set of preview images includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, where the second frame rate is less than the first frame rate. Upon receiving a shooting command, the processing module generates a second image based on the second set of preview images.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program / program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, based on the exposure information corresponding to the first shooting scene, a first preview image set is acquired. The exposure information includes a first long exposure duration and a first short exposure duration. The first preview image set includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. A first image is displayed based on the image frames in the first preview image set, and a second preview image set is cached. The second preview image set includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, where the second frame rate is less than the first frame rate. Upon receiving a shooting command, a second image is generated based on the second preview image set. In this solution, the electronic device can dynamically determine the long exposure duration and short highlight duration corresponding to the current shooting scene, and acquire a high frame rate image frame set through the long exposure duration and short highlight duration. Then, the first image corresponding to the long exposure image frame is displayed, thus ensuring the image quality of the preview image. Furthermore, upon receiving a shooting command, the second image is obtained by image processing through a low frame rate image frame set, which can reduce the power consumption of the electronic device when shooting images in high frame rate mode. Attached Figure Description

[0013] Figure 1 This is one of the flowcharts of a shooting method provided in the embodiments of this application;

[0014] Figure 2 This is one of the schematic diagrams of a high frame rate path example provided in the embodiments of this application;

[0015] Figure 3 This is a second flowchart of a shooting method provided in an embodiment of this application;

[0016] Figure 4 This is the third flowchart of a shooting method provided in the embodiments of this application;

[0017] Figure 5 This is the fourth flowchart of a shooting method provided in the embodiments of this application;

[0018] Figure 6 This is a second schematic diagram of a high frame rate path example provided in the embodiments of this application;

[0019] Figure 7 This is the third schematic diagram of a high frame rate path example provided in the embodiments of this application;

[0020] Figure 8 This is a schematic diagram of the structure of a shooting device provided in an embodiment of this application;

[0021] Figure 9 This is one of the hardware structure diagrams of an electronic device provided in the embodiments of this application;

[0022] Figure 10 This is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects. For example, a first object can be one or more, where "more" means at least two. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] The terms "at least one" and "at least one of" in this application's specification refer to any one, any two, or a combination of two or more of the included objects. For example, "at least one of a, b, and c" can mean "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple, and multiple means at least two. Similarly, "at least two" means two or more, and its meaning is similar to "at least one". The identifiers in this application are text, symbols, images, etc., used to indicate information, and can use controls or other containers as carriers for displaying information, including but not limited to text identifiers, symbol identifiers, and image identifiers.

[0026] The shooting method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0027] The shooting method provided in this application can be applied to snapshot scenarios. For example, it can be used to capture athletes on a basketball court or a football field.

[0028] Currently, with the rapid development of smartphones, photography has become a very important application scenario, and shooting moving objects occupies a significant proportion of these scenarios. Professional cameras are also using motion capture as a selling point. Currently, the preview frame rate for snapshot modes in most smartphones is 30fps, far lower than the 120fps high frame rate preview of professional cameras.

[0029] In related technologies, when shooting objects moving at extremely high speeds, such as speeding race cars, if the frame rate is maintained at 30fps, a preview frame is generated every 33ms before being used by subsequent focus tracking or autofocus algorithms. In this scenario, by the time the motor is aligned with the position of the previous frame, the subject has already moved a significant distance in the next frame, leading to focusing problems. Furthermore, the preview smoothness at 30fps is not good enough when viewing moving objects.

[0030] Current mobile phone motion capture solutions all use image sensors that simultaneously output two frames of data in a long-exposure and short-exposure manner. The long exposure ensures image quality, while the short exposure ensures the frame is captured. At a preview frame rate of 30fps, the actual image sensor outputs 60 frames per second. If the preview frame rate is to be increased to 60fps, the actual output of the image sensor would be 120fps. This poses a significant challenge to the performance and power consumption of electronic devices.

[0031] Based on the scenarios described above in the embodiments of this application, the shooting method provided by the embodiments of this application allows the electronic device to dynamically determine the long exposure duration and short highlight duration corresponding to the current shooting scene, and to acquire a set of high frame rate image frames through the long exposure duration and short highlight duration. Then, the first image corresponding to the long exposure image frame is displayed, thus ensuring the image quality of the preview image. Furthermore, upon receiving a shooting instruction, the second image is obtained by image processing through a set of low frame rate image frames, which can reduce the power consumption of the electronic device when shooting images in high frame rate mode.

[0032] The subject of the shooting method provided in this application is a shooting device, which can be an electronic device, or a functional module or entity within an electronic device. This application does not limit the specific device to this type of shooting device. The following will use an electronic device as an example to illustrate the shooting method provided in this application.

[0033] This application provides a shooting method. Figure 1 A flowchart illustrating a shooting method provided in an embodiment of this application is shown. Figure 1 As shown, the shooting method provided in this application embodiment may include the following steps 201 to 203.

[0034] Step 201: The electronic device collects a first set of preview images based on the exposure information corresponding to the first shooting scene.

[0035] In this embodiment of the application, the exposure information includes a first long exposure duration and a first short exposure duration, and the first preview image set includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate.

[0036] Optionally, in this embodiment of the application, the first shooting scene can be a moving scene or a non-moving scene.

[0037] Optionally, in this embodiment of the application, the electronic device can collect a first set of preview images based on the exposure information corresponding to the first shooting scene in a first shooting mode and a first shooting scene.

[0038] For example, the first shooting mode described above can be a snapshot shooting mode.

[0039] For example, the first frame rate mentioned above can be 60 frames.

[0040] Optionally, in the embodiments of this application, the long exposure image frame can be a long exposure original image frame, and the short exposure image frame can be a short exposure original image frame.

[0041] Optionally, in this embodiment of the application, the electronic device may acquire a first set of preview images via an image sensor.

[0042] For example, the image sensor described above can be a charge-coupled device (CCD) image sensor or a complementary metal-oxide-semiconductor (CMOS) image sensor. The specific sensor can be determined based on actual usage requirements, and this application does not impose any limitations.

[0043] For example, when there are multiple long exposure image frames, each long exposure image frame corresponds to a short exposure image frame.

[0044] For example, when a user triggers an electronic device to run a camera application and enters motion capture mode, the electronic device can switch the image sensor's output mode from 30fps single exposure mode to 60fps double exposure mode, switch the image processing path from low frame rate path to high frame rate path, and acquire long exposure image frames and corresponding short exposure image frames through exposure information.

[0045] Optionally, in this embodiment of the application, the electronic device can configure the Auto Exposure (AE) algorithm, AF algorithm, motion detection algorithm, scene detection algorithm, etc., to a high frame rate mode, i.e., 60fps mode.

[0046] Step 202: The electronic device displays the first image based on the image frames in the first preview image set, and caches the second preview image set.

[0047] In this embodiment of the application, the second preview image set includes long exposure image frames at the second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, wherein the second frame rate is less than the first frame rate.

[0048] For example, the second frame rate mentioned above can be 30 frames.

[0049] Optionally, in this embodiment of the application, the electronic device may display the first image in the shooting preview interface.

[0050] It can be understood that the first image mentioned above is obtained by the electronic device through image processing of image frames in the first preview image set.

[0051] Optionally, in this embodiment of the application, the electronic device may cache the second set of preview images in the cache pool corresponding to the high frame rate path.

[0052] For example, such as Figure 2 As shown, the high frame rate path described above may include an image sensor 10. Figure 2 The image sensor 10, referred to as "sensor" in the text, can be connected to both the buffer pool 11 and the image processor 12. Figure 2 The image processor 12, referred to as ISP, can be connected to the preview display module 13. After acquiring 60 long-exposure and short-exposure image frames, the image sensor 10 obtains 30 long-exposure and short-exposure image frames, caches these 30 frames in a buffer pool 11, and inputs the 60 long-exposure and short-exposure image frames into the image processor 12. The image processor 12 then outputs a first image to the preview display module 13 based on these 60 long-exposure and short-exposure image frames, allowing the preview display module 13 to display the first image.

[0053] Optionally, in this embodiment of the application, the above-mentioned buffer pool may include two independent circular buffers, one circular buffer for storing 30 long exposure image frames and the other circular buffer for storing 30 short exposure image frames.

[0054] It should be noted that although the two independent circular buffers are physically stored independently, logically each pair of long-exposure image frames and short-exposure image frames acquired within the same time period are associated through timestamps or index pointers to ensure that subsequent processing can be correctly paired.

[0055] Optionally, in this embodiment of the application, the total capacity of the above-mentioned buffer pool is designed to store N pairs of exposed image frames, for example, N=15, and the setting of the capacity N can be determined based on the first condition.

[0056] For example, the first condition mentioned above can be: it can accommodate the data continuously generated by the image processor from the time the user presses the shutter until the frame capture is completed without losing frames, and can capture the image seen in the user's preview, while avoiding excessive use of system memory resources.

[0057] Optionally, in this embodiment of the application, the above-mentioned cache pool may involve two parallel data transmission processes.

[0058] For example, the two parallel data transmission processes described above can be a data writing process and a data reading process.

[0059] For example, the data writing process described above can be as follows: When the sum of the first long exposure duration and the first short exposure duration is less than 16.6 ms, the image processor can alternately output long exposure frames and short exposure frames at a maximum rate of 120 fps. The image processor's input interface or a dedicated Direct Memory Access Controller (DMAC) continuously writes the long exposure data stream and the short exposure data stream into their respective buffers. The writing process follows a "first-in, first-out" (FIFO) principle. When the buffer is full, new data will overwrite the oldest data.

[0060] For example, the data reading process described above can be as follows: The electronic device can read data from the long exposure frame buffer only at a fixed 60fps. After the read data undergoes noise reduction and color correction by the ISP, it is sent to the screen for display, providing the user with a high-quality 60fps real-time preview. Alternatively, when the user triggers a photo capture command, the system simultaneously locks the long exposure and corresponding short exposure frame data based on the timestamp of the preview frame at the moment of capture and the number of frames required by the algorithm. These locked data frames are then sent to a multi-frame synthesis algorithm for deep fusion processing to generate the final high-quality photo.

[0061] Optionally, in this embodiment of the application, the cache pool may involve cache invalidation.

[0062] For example, the cache invalidation mentioned above can include natural invalidation and active clearing.

[0063] For example, the aforementioned natural failure can be described as follows: In normal preview mode, due to limited cache capacity, when new data is written, the oldest unconsumed data is automatically overwritten; this is natural failure. This ensures that the cache always contains the most recent image data.

[0064] The aforementioned proactive clearing can be achieved by the electronic device proactively clearing the entire cache pool when switching cameras or exiting the shooting mode, in order to avoid leaving irrelevant image data.

[0065] It should be noted that the process by which the electronic device obtains the second set of preview images can be found in the following embodiments, and will not be repeated here to avoid repetition.

[0066] Step 203: Upon receiving a shooting instruction, the electronic device generates a second image based on the second set of preview images.

[0067] Optionally, in this embodiment of the application, the shooting command can be voice input or click input.

[0068] For example, an electronic device can receive a shooting instruction based on the user's input of clicking on the shooting control.

[0069] Optionally, in this embodiment, the electronic device can simultaneously lock long-exposure and corresponding short-exposure frame data based on the timestamp of the preview display frame at the time of taking the picture and the number of frames required by the algorithm. These locked data frames will be sent to a multi-frame synthesis algorithm for deep fusion processing to generate the final high-quality photo, i.e., the second image.

[0070] This application provides a shooting method. Based on exposure information corresponding to a first shooting scene, a first preview image set is acquired. The exposure information includes a first long exposure duration and a first short exposure duration. The first preview image set includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. A first image is displayed based on the image frames in the first preview image set, and a second preview image set is cached. The second preview image set includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, where the second frame rate is less than the first frame rate. Upon receiving a shooting command, a second image is obtained based on the second preview image set. In this solution, the electronic device can dynamically determine the long exposure duration and short highlight duration corresponding to the current shooting scene, and acquire a high frame rate image frame set using the long exposure duration and short highlight duration. Then, the first image corresponding to the long exposure image frames is displayed, thus ensuring the image quality of the preview image. Furthermore, upon receiving a shooting command, the second image is obtained by image processing using a low frame rate image frame set, which can reduce the power consumption of the electronic device when shooting images in high frame rate mode.

[0071] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 3 As shown, before step 201 above, the shooting method provided in this application embodiment further includes steps 301 to 303 as described below.

[0072] Step 301: When the preview interface of the first shooting mode is displayed, the electronic device determines the first shooting scene in which the electronic device is located based on the scene information in the first shooting mode.

[0073] In this embodiment of the application, the scene information includes the motion speed information of the subject being photographed and the brightness information of the shooting environment.

[0074] Optionally, in this embodiment of the application, the electronic device can determine the first shooting scene in which the electronic device is located through a scene detection algorithm.

[0075] For example, the first shooting scene described above can be a basketball court, a football field, a stadium, etc. The specific location can be determined according to actual usage needs, and this application embodiment does not impose any limitations.

[0076] Step 302: When the first shooting scene is a motion shooting scene, determine the weight of long exposure time based on the shooting environment brightness information, and determine the weight of short exposure time based on the motion speed information of the subject.

[0077] Optionally, in this embodiment of the application, the electronic device can determine that the first shooting scene is a moving shooting scene through a motion detection algorithm.

[0078] Optionally, in this embodiment of the application, the electronic device can obtain ambient brightness information through a brightness detection module.

[0079] For example, the brightness detection module described above can be a photoresistor module.

[0080] Optionally, in this embodiment, the aforementioned ambient brightness information can be positively correlated with the weight of long exposure time. That is, the higher the ambient brightness information, the greater the weight of long exposure time.

[0081] Optionally, in this embodiment of the application, the subject of the photograph can be a person or an animal.

[0082] Optionally, in this embodiment of the application, the electronic device can obtain the motion speed information of the subject being photographed based on the vector motion detection module.

[0083] For example, the vector motion detection module described above can be any of the following: an LD2410B human body sensing module, an OpenCV motion detection and tracking framework, or a CPLD digital video signal motion detection module, etc. The specific module can be determined according to actual usage requirements, and this application embodiment does not impose any limitations.

[0084] Optionally, in this embodiment, the motion speed information can be positively correlated with the short exposure duration weight. That is, the higher the motion speed information, the greater the weight of the short exposure duration.

[0085] Step 303: The electronic device determines the first long exposure time based on the long exposure time weight and the preset long exposure time corresponding to the first shooting mode, and determines the first short exposure time based on the short exposure time weight and the preset short exposure time corresponding to the first shooting mode.

[0086] Optionally, in this embodiment of the application, the electronic device can weight the long exposure duration with the preset long exposure duration corresponding to the first shooting mode to obtain the first long exposure duration.

[0087] For example, the electronic device can multiply the preset long exposure duration corresponding to the first shooting mode with the long exposure duration weight to obtain the first long exposure duration.

[0088] Optionally, in this embodiment of the application, the electronic device can weight the short exposure duration with the preset short exposure duration corresponding to the first shooting mode to obtain the first short exposure duration.

[0089] For example, the electronic device can multiply the preset short exposure duration corresponding to the first shooting mode with the short exposure duration weight to obtain the first short exposure duration.

[0090] In this embodiment, the electronic device can dynamically determine the long exposure duration and short exposure duration based on the motion speed information of the subject being photographed in the current shooting scene and the brightness information of the shooting environment, thereby improving the flexibility of the electronic device in determining the long exposure duration and short exposure duration.

[0091] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 4 As shown, step 201 can be implemented through steps 201a and 201b below.

[0092] Step 201a: The electronic device sums the first long exposure time and the first short exposure time to obtain the first exposure time.

[0093] For example, the electronic device can add the first long exposure time and the first short exposure time to obtain the first exposure time.

[0094] Step 201b: When the first exposure duration is less than the first threshold, the electronic device acquires a long exposure image frame based on the first long exposure duration, and acquires a short exposure image frame corresponding to the long exposure image frame based on the first short exposure duration.

[0095] In this embodiment of the application, the first threshold is the duration of displaying one frame of image at the first frame rate.

[0096] For example, the first threshold mentioned above can be 16.6ms.

[0097] For example, the process of acquiring the first set of preview images provided in this application will be explained in detail below through specific embodiments.

[0098] 1. Exposure control process in motion scenarios:

[0099] When an electronic device determines that the current scene is a "moving scene" through scene recognition or motion detection algorithms, it will trigger a specific exposure strategy.

[0100] The specific exposure strategy can be: the AE algorithm obtains a scene type identifier, which is used to indicate whether it is a moving scene; motion information, such as the speed, amplitude or trajectory of the subject's movement; ambient brightness information: the image information of the current original image frame; and algorithm processing capability: the upper limit of the image quality processing of the backend multi-frame image processing algorithm.

[0101] Then, the exposure time is dynamically calculated using the AE algorithm: the core task of the AE algorithm is to calculate a set of optimal long and short exposure times. Its calculation strategy is as follows:

[0102] Objective trade-off: The AE algorithm needs to strike an optimal balance between "image quality of long exposure frames" and "instantaneous capture capability of short exposure frames." Long exposure frames are used to ensure image signal-to-noise ratio and overall brightness, while short exposure frames are used to freeze the moment of motion and avoid motion blur.

[0103] Dynamic adjustment strategy:

[0104] Motion speed weighting: The detected motion speed is the primary factor in determining the short exposure time. The faster the motion speed, the higher the weight assigned, and the algorithm will prioritize calculating a shorter short exposure time, for example, less than 1 / 1000 of a second, to ensure that fast-moving subjects can be clearly captured.

[0105] Ambient brightness weighting: Ambient brightness is a key factor in determining long exposure time and overall exposure. In low-light environments, to ensure sufficient light intake for long exposure frames, the weight and duration of the long exposure time can be appropriately increased.

[0106] Multi-factor fusion calculation: The AE algorithm uses a weighted function to comprehensively process the above factors.

[0107] Hard constraint: In motion scenes, to ensure smooth previewing, a hard constraint is set: the sum of the long exposure time and the short exposure time must be less than the theoretical time to display one frame, for example, 16.6ms, corresponding to a preview frame rate of 60 frames per second. The algorithm optimizes the allocation of long and short exposure times while satisfying this constraint.

[0108] Output and Execution: The AE algorithm sends the calculated long exposure time and short exposure time to the image sensor. The image sensor then sequentially acquires long exposure frames and short exposure frames, which are then fused by the backend algorithm.

[0109] 2. Exposure control process in non-moving scenes:

[0110] When the system determines that the current scene is not a motion scene, the AE algorithm adopts the standard strategy.

[0111] Input parameters: mainly based on ambient brightness information and algorithm processing capabilities.

[0112] Computational Logic: The algorithm prioritizes improving overall image quality and signal-to-noise ratio, no longer limited by the need for motion freezing. Therefore, it can calculate longer combinations of exposure times, meaning both long and short exposure times can be extended.

[0113] Adaptive frame rate: In this case, the sum of the long and short exposure times may exceed 16.6ms. The system can dynamically reduce the preview frame rate, for example, from 60 frames / second to 50, 40, or 30 frames / second, to accommodate longer exposure times and ensure image quality.

[0114] In this embodiment, the electronic device can dynamically calculate the exposure time to control the dynamic change of the preview frame rate, thereby achieving a preview effect of up to 60fps in the snapshot scene.

[0115] Optionally, in the embodiments of this application, combined with Figure 1 ,like Figure 5 As shown, prior to step 202 above, the shooting method provided in this application embodiment further includes steps 401 and 402 as described below.

[0116] Step 401: The electronic device determines the frame dropping interval based on the first frame rate and the second frame rate.

[0117] Optionally, in this embodiment of the application, the electronic device may perform a division operation on the first frame rate and the second frame rate to obtain the frame dropping interval.

[0118] Step 402: The electronic device performs frame dropping processing on the image frames in the first preview image set based on the frame dropping interval to obtain the second preview image set at the second frame rate.

[0119] Optionally, in this embodiment of the application, the electronic device can delete long-exposure image frames and corresponding short-exposure image frames in the first preview image set by dropping frame intervals to obtain a second preview image set at the second frame rate.

[0120] In this embodiment, the electronic device performs frame dropping processing on the first preview image set to obtain a second preview image set at the second frame rate. This allows the electronic device to obtain image frames from the second preview image set at the low frame rate when it receives a shooting command, thereby reducing the power consumption of the electronic device in the high frame rate shooting mode.

[0121] Optionally, in the embodiments of this application, step 202 can be implemented by step 202a or step 202b as described below.

[0122] Step 202a: The electronic device performs image processing on the long exposure image frames in the first preview image set to obtain and display the first image, and caches the second preview image set.

[0123] In this embodiment of the application, the image resolution of the first image is equal to the first frame rate.

[0124] For example, the image resolution of the first image described above can be 60 frames per second.

[0125] Optionally, in this embodiment of the application, the electronic device can perform AE, AF and autofocus processing on the long exposure image frames in the first preview image set through an image processor to obtain the first image.

[0126] For example, combined Figure 2 ,like Figure 6 As shown, the image processor 12 can output the focus statistics required by the AF algorithm, the exposure statistics required by the AE algorithm, and a low-resolution (e.g., 640*480, YUV) small image required by the tracking focus algorithm, all at a frame rate of 60fps. For the same frame, the three algorithms process in parallel. For different frames, the AF algorithm uses the result output from the previous frame's tracking focus algorithm during calculation. Each algorithm must complete processing within 16.6ms; otherwise, it will cause dropped preview frames.

[0127] Thus, 60fps doubles the sampling frequency compared to 30fps, resulting in more timely response during fast motion, tighter subject focus tracking, and more solid focus engagement. When the preview frame rate is increased from 30fps to 60fps, assuming a high-speed moving object scene, the frame interval is 100 pixels / frame at 30fps, but at 60fps, the frame interval can be halved to 50 pixels / frame. The AE algorithm typically requires 5-10 frames for convergence. Taking 10 frames as an example, it takes 330ms at 30fps, but half that at 60fps, it only takes 165ms.

[0128] Step 202b: The electronic device performs image processing on the long exposure image frames and short exposure image frames in the first preview image set to obtain and display the first image, and caches the second preview image set.

[0129] In this embodiment of the application, the image resolution of the first image is greater than the first frame rate.

[0130] For example, the image resolution of the first image described above can be 120 frames per second.

[0131] Optionally, in the embodiments of this application, step 202b can be implemented by steps 202b1 to 202b3 as described below.

[0132] Step 202b1: The electronic device processes the long exposure image frame and the corresponding short exposure image frame in the first preview image set through the image processor to obtain the first long exposure image frame and the corresponding first short exposure image frame.

[0133] It should be noted that the process by which the electronic device processes the long exposure image frames and the corresponding short exposure image frames in the first preview image set through the image processor can be found in the above embodiments, and will not be repeated here to avoid repetition.

[0134] Step 202b2: The electronic device obtains the tonal information of the first long exposure image frame through the tonal transfer model, and maps the tonal information into the first short exposure image frame to obtain the second short exposure image frame.

[0135] Optionally, in this embodiment of the application, the electronic device can perform tonal information feature extraction processing on the first long exposure image frame through a tonal transfer model to obtain tonal information, and apply the tonal information to the first short exposure image frame through the tonal transfer model to obtain the second short exposure image frame.

[0136] Step 202b3: The electronic device performs image fusion processing on the first long exposure image frame and the second short exposure image frame to obtain and display the first image, and caches the second preview image set.

[0137] For example, combined Figure 2 ,like Figure 7 As shown, the high frame rate path described above may further include a tone learning and transfer module 20. The electronic device inputs 60 long exposure image frames and short exposure image frames into the image processor 12, so that the image processor 12 processes the 60 long exposure image frames and short exposure image frames to obtain a first long exposure image frame and a corresponding short exposure image frame. Then, the first long exposure image frame and the corresponding short exposure image frame are input into the tone learning and transfer module 20. The tone learning and transfer module 20 transfers the tone information in the first long exposure image frame to the corresponding short exposure image frame to obtain the first short exposure image frame. Then, the first long exposure image frame and the first short exposure image frame are input into the preview display module 13. The preview display module 13 performs image fusion on the first long exposure image frame and the first short exposure image frame to obtain a first image of 120 frames and displays the first image.

[0138] In this embodiment, a tone learning and transfer algorithm is added after the image processor processing. When a long exposure image frame is sent, the algorithm learns the tone effect of the long exposure image frame. When a short exposure frame is sent, the learned tone is transferred to the short exposure to ensure that the tone effect of the short exposure frame is aligned with the long exposure frame. In this way, the tone does not jump when long and short exposures are sent alternately.

[0139] Optionally, in the embodiments of this application, the shooting method provided in the embodiments of this application further includes the following steps 501 to 505.

[0140] Step 501: The electronic device acquires the training image set and the reference image frame.

[0141] In this embodiment of the application, the above-mentioned training image frame set includes long exposure training image frames and short exposure training image frames.

[0142] Optionally, in the embodiments of this application, both the long exposure training image frames and the short exposure training image frames can be images processed by an image processor.

[0143] Optionally, in the embodiments of this application, the aforementioned long exposure training image frames can be one or more, and the aforementioned short exposure training image frames can also be one or more.

[0144] It should be noted that the long exposure training image frames and short exposure training image frames mentioned above can be image frames with the same image content but different exposure durations.

[0145] Step 502: The electronic device inputs the training image set and reference image frames into the initial model, performs tonal feature extraction processing on the long exposure training image frames, and obtains tonal feature information.

[0146] Optionally, in this embodiment of the application, the initial model may include a tonal feature extraction module and a tonal transfer module.

[0147] For example, the above-mentioned tone feature extraction module can be a convolutional neural network with an encoder structure, and the tone feature extraction module can include multiple convolutional layers and multiple pooling layers.

[0148] It should be noted that one convolutional layer can correspond to one pooling layer. The number of convolutional and pooling layers can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions.

[0149] For example, an electronic device can input a training image set and a reference image frame into a tone feature extraction module to output a low-dimensional tone feature vector through the tone feature extraction module.

[0150] It should be noted that the aforementioned low-dimensional tonal feature vectors may include advanced tonal attributes such as global color distribution, contrast, and brightness style of long-exposure training image frames, rather than specific image content.

[0151] Step 503: The electronic device maps the tonal feature information onto the short exposure training image frame through the initial model to obtain the third short exposure image frame.

[0152] Optionally, in this embodiment of the application, the electronic device can use a tone transfer module to change the tone feature information in the short exposure training image to the tone feature information in the long exposure image to obtain a third short exposure image frame.

[0153] For example, the tonal transfer module described above can generate a network for U-Net that includes skip connections.

[0154] Step 504: The electronic device calculates a set of loss values ​​based on the third short-exposure image frame and the reference image frame.

[0155] In this embodiment of the application, the above-mentioned loss value set is used to characterize the differences in image texture, image brightness, and image noise between the third short-exposure image frame and the reference image frame.

[0156] In this embodiment, the electronic device can perform cross-entropy loss calculation on the image information in the third short-exposure image frame and the image information in the reference image frame to obtain a set of loss values.

[0157] It should be noted that the specific implementation process of step 504 above can be found in the following embodiments, and will not be repeated here to avoid repetition.

[0158] Step 505: The electronic device trains the initial model based on the loss value set to obtain the tone transfer model.

[0159] Optionally, in this embodiment of the application, the electronic device can train the initial model using a set of loss values ​​and backpropagation to obtain a tone transfer model.

[0160] In this embodiment, the electronic device adds a tone learning and transfer module after the image processor to keep the tone of the short exposure frame, such as color style, contrast, and overall hue, consistent with that of the long exposure frame. This can eliminate the visual jumps caused by the alternating display of long and short exposure frames and improve image quality.

[0161] Optionally, in the embodiments of this application, the above-mentioned loss value set includes a first loss value, a second loss value, and a third loss value. The first loss value is used to characterize the image texture difference between the third short exposure image frame and the reference image frame, the second loss value is used to characterize the image brightness difference between the third short exposure image frame and the reference image frame, and the third loss value is used to characterize the image noise difference between the third short exposure image frame and the reference image frame.

[0162] For example, step 504 above can be implemented by steps 504a to 504c below.

[0163] Step 504a: The electronic device performs cross-entropy processing on the image texture feature information of the third short-exposure image frame and the image texture feature information of the reference image frame to obtain the first loss value.

[0164] Optionally, in this embodiment of the application, the above-mentioned image texture feature information may include at least one of the following:

[0165] It should be noted that the specific implementation process of step 504a above can be found in the description in the relevant technology. To avoid repetition, it will not be repeated here.

[0166] Step 504b: The electronic device performs cross-entropy processing on the image brightness feature information of the third short-exposure image frame and the image brightness feature information of the reference image frame to obtain the second loss value.

[0167] It should be noted that the specific implementation process of step 504b above can be found in the description in the relevant technology. To avoid repetition, it will not be repeated here.

[0168] Step 504c: The electronic device performs cross-entropy processing on the image noise feature information of the third short-exposure image frame and the image noise feature information of the reference image frame to obtain the third loss value.

[0169] It should be noted that the specific implementation process of step 504c above can be found in the description in the relevant technology. To avoid repetition, it will not be repeated here.

[0170] By way of example, the model training process provided in the embodiments of this application will be explained in detail below.

[0171] Core objective: To achieve real-time consistency of tonal values ​​between frames. Specifically, when a long-exposure image frame is input, the algorithm learns its tonal features; when the next short-exposure image frame is input, the algorithm transfers the learned long-exposure tonal features to the short-exposure frame, generating a new image whose tonal value is aligned with the long-exposure frame, and then outputs it.

[0172] Workflow: The algorithm processes alternating long and short exposure frame streams in a ping-pong manner, forming a real-time "learning-transfer-display" loop.

[0173] Model Structure and Network Design: Considering the stringent performance requirements of mobile deployment, namely, processing must be completed within milliseconds, this algorithm adopts a carefully designed lightweight convolutional neural network, whose structure mainly includes two core components: a tone feature extraction module and a tone transfer module.

[0174] The input to the tonal feature extraction module is a single RGB long exposure image frame that has undergone basic ISP processing.

[0175] The structure of the tone feature extraction module is: a CNN with an encoder structure, consisting of several convolutional layers and pooling layers.

[0176] The output of the tonal feature extraction module is a low-dimensional tonal feature vector. This vector encodes high-level tonal attributes of the input image, such as global color distribution, contrast, and brightness style, rather than the specific image content.

[0177] The input to the tone transfer network is a short exposure image frame corresponding to a long exposure image frame.

[0178] The structure of the tone transfer network is: a lightweight U-Net or similar generative network containing skip connections to preserve image details (especially sharp edges of moving objects) in short exposure frames.

[0179] The output of the tone transfer network is a short-exposure frame image after tone adjustment.

[0180] Training data sources: Offline training was conducted using large-scale professional datasets. Data sources included publicly available image datasets, such as MIT-Adobe FiveK, which contained various tonal effects of the same scene adjusted by professional photographers.

[0181] Internally acquired paired data: Two images are simultaneously captured in the same scene using long and short exposure times, and then color-graded by professional image engineers to match the tonal range of the short-exposure image with that of the long-exposure image. This constitutes the most direct supervisory signal.

[0182] Loss function: Model training is optimized by combining loss functions to ensure the quality of transfer effect: Tonal loss: Calculate the distance between the generated image and the short-exposure target image in the tonal feature space, such as using Gram matrix loss or feature loss based on pre-trained VGG network, to force tonal alignment.

[0183] Content preservation loss: Using perceptual loss or L1 / L2 loss, the structure and details of the generated image are kept consistent with the original short-exposure input image, avoiding over-smoothing or distortion.

[0184] Total variational loss: encourages spatial smoothness in generated images and reduces unnatural noise.

[0185] The online transfer strategy is a "learning-transfer" ping-pong mechanism: N long exposure image frames are input into the tonal feature extraction network, and the calculated tonal feature vectors are temporarily stored in a buffer register, where N is a positive integer.

[0186] Short exposure frames: Input the short exposure image frames corresponding to the N long exposure image frames into the model. The tone transfer module simultaneously receives the short exposure frames and the long exposure tone features in the buffer register, and generates tone-consistent output frames in real time.

[0187] This process is repeated continuously to achieve dynamic tonal matching for each pair of long and short exposure frames.

[0188] In this embodiment, the model learns the essential representation of "tone" and can generalize it to unseen scenes. During online runtime, tone transfer can be achieved without retraining or complex calculations, effectively eliminating flickering and abrupt changes in the preview image.

[0189] It should be noted that the above-described method embodiments, or the various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0190] It should be noted that the shooting method provided in this application embodiment can be executed by a shooting device. This application embodiment uses a shooting device executing the shooting method as an example to illustrate the shooting device provided in this application embodiment.

[0191] Figure 8 A schematic diagram of a possible structure of the imaging device involved in an embodiment of this application is shown. For example... Figure 8 As shown, the imaging device 70 may include an acquisition module 71 and a processing module 72.

[0192] The acquisition module 71 is used to acquire a first set of preview images based on the exposure information corresponding to the first shooting scene. The exposure information includes a first long exposure duration and a first short exposure duration. The first set of preview images includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. The processing module 72 is used to display a first image based on the image frames in the first set of preview images and to cache a second set of preview images. The second set of preview images includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, where the second frame rate is less than the first frame rate. Upon receiving a shooting command, the processing module 72 generates a second image based on the second set of preview images.

[0193] In one possible implementation, the processing module 72 is further configured to, before the acquisition module acquires the first set of preview images based on the exposure information corresponding to the first shooting scene when the electronic device is in the first shooting scene, determine the first shooting scene where the electronic device is located based on the scene information in the first shooting mode, while displaying the preview interface of the first shooting mode, the scene information including the motion speed information of the shooting subject and the brightness information of the shooting environment; if the first shooting scene is a motion shooting scene, determine the weight of the long exposure time based on the brightness information of the shooting environment, and determine the weight of the short exposure time based on the motion speed information of the shooting subject; determine the first long exposure time based on the weight of the long exposure time and the preset long exposure time corresponding to the first shooting mode, and determine the first short exposure time based on the weight of the short exposure time and the preset short exposure time corresponding to the first shooting mode.

[0194] In one possible implementation, the processing module 72 is specifically used to sum the first long exposure duration and the first short exposure duration to obtain the first exposure duration. The acquisition module 71 is specifically used to acquire a long exposure image frame based on the first long exposure duration, and to acquire a short exposure image frame corresponding to the long exposure image frame based on the first short exposure duration, when the first exposure duration is less than a first threshold; wherein, the first threshold is the duration of displaying one frame of image at the first frame rate.

[0195] In one possible implementation, the processing module 72 is further configured to determine the frame dropping interval based on the first frame rate and the second frame rate before caching the second preview image set; and to perform frame dropping processing on the image frames in the first preview image set based on the frame dropping interval to obtain the second preview image set at the second frame rate.

[0196] In one possible implementation, the processing module 72 is specifically used to perform image processing on long-exposure image frames in the first preview image set to obtain and display a first image, wherein the image resolution of the first image is equal to the first frame rate. Alternatively, image processing is performed on long-exposure image frames and short-exposure image frames in the first preview image set to obtain and display a first image, wherein the image resolution of the first image is greater than the first frame rate.

[0197] In one possible implementation, the processing module 72 is specifically used to process the long exposure image frame and the corresponding short exposure image frame in the first preview image set through an image processor to obtain a first long exposure image frame and a corresponding first short exposure image frame; obtain the tonal information of the first long exposure image frame through a tonal transfer model, and map the tonal information into the first short exposure image frame to obtain a second short exposure image frame; and perform image fusion processing on the first long exposure image frame and the second short exposure image frame to obtain and display the first image.

[0198] In one possible implementation, the imaging device 70 provided in the above embodiments of this application further includes: an acquisition module. The acquisition module is used to acquire a training image set and a reference image frame, the training image frame set including long-exposure training image frames and short-exposure training image frames. The processing module 72 is further used to input the training image set and the reference image frame into an initial model; to perform tonal feature extraction processing on the long-exposure training image frame to obtain tonal feature information; to map the tonal feature information onto the short-exposure training image frame through the initial model to obtain a third short-exposure image frame; to calculate a loss value set based on the third short-exposure image frame and the reference image frame, the loss value set being used to characterize the image texture difference, image brightness difference, and image noise difference between the third short-exposure image frame and the reference image frame; and to train the initial model based on the loss value set to obtain a tonal transfer model.

[0199] In one possible implementation, the aforementioned set of loss values ​​includes a first loss value, a second loss value, and a third loss value. The first loss value characterizes the image texture difference between the third short-exposure image frame and the reference image frame; the second loss value characterizes the image brightness difference between the third short-exposure image frame and the reference image frame; and the third loss value characterizes the image noise difference between the third short-exposure image frame and the reference image frame. Specifically, the processing module 72 performs cross-entropy processing on the image texture feature information of the third short-exposure image frame and the image texture feature information of the reference image frame to obtain the first loss value; performs cross-entropy processing on the image brightness feature information of the third short-exposure image frame and the image brightness feature information of the reference image frame to obtain the second loss value; and performs cross-entropy processing on the image noise feature information of the third short-exposure image frame and the image noise feature information of the reference image frame to obtain the third loss value.

[0200] In the shooting device provided in this application embodiment, the shooting device can dynamically determine the long exposure duration and short highlight duration corresponding to the current shooting scene according to the current shooting scene, and acquire a set of high frame rate image frames through the long exposure duration and short highlight duration. Then, it displays the first image corresponding to the long exposure image frame, which can ensure the image quality of the preview image. Furthermore, when a shooting command is received, the second image can be obtained by image processing through a set of low frame rate image frames, which can reduce the power consumption of the shooting device when shooting images in high frame rate mode.

[0201] The shooting device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope.

[0202] The shooting device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0203] The imaging device provided in this application embodiment can realize all the processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0204] Optionally, such as Figure 9 As shown, this application embodiment also provides an electronic device 90, including a processor 91 and a memory 92. The memory 92 stores a program or instructions that can run on the processor 91. When the program or instructions are executed by the processor 91, they implement the various steps of the above-described shooting method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0205] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0206] Figure 10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0207] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0208] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0209] The processor 110 is configured to acquire a first preview image set based on exposure information corresponding to a first shooting scene. The exposure information includes a first long exposure duration and a first short exposure duration. The first preview image set includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. The processor 110 is configured to display a first image based on the image frames in the first preview image set and cache a second preview image set. The second preview image set includes long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, where the second frame rate is less than the first frame rate. Upon receiving a shooting command, the processor 110 is configured to generate a second image based on the second preview image set.

[0210] Optionally, in this embodiment of the application, the processor 110 is further configured to, when the electronic device is in a first shooting scene, before acquiring a first set of preview images based on the exposure information corresponding to the first shooting scene, determine the first shooting scene in which the electronic device is located based on the scene information in the first shooting mode, wherein the scene information includes the motion speed information of the shooting subject and the brightness information of the shooting environment; when the first shooting scene is a motion shooting scene, determine the weight of the long exposure time based on the brightness information of the shooting environment, and determine the weight of the short exposure time based on the motion speed information of the shooting subject; determine the first long exposure time based on the weight of the long exposure time and the preset long exposure time corresponding to the first shooting mode, and determine the first short exposure time based on the weight of the short exposure time and the preset short exposure time corresponding to the first shooting mode.

[0211] Optionally, in this embodiment of the application, the processor 110 is specifically used to sum the first long exposure duration and the first short exposure duration to obtain the first exposure duration; when the first exposure duration is less than the first threshold, to acquire a long exposure image frame based on the first long exposure duration, and to acquire a short exposure image frame corresponding to the long exposure image frame based on the first short exposure duration; wherein, the first threshold is the duration of displaying one frame of image at the first frame rate.

[0212] Optionally, in this embodiment of the application, the processor 110 is further configured to determine the frame dropping interval based on the first frame rate and the second frame rate before caching the second preview image set; and to perform frame dropping processing on the image frames in the first preview image set based on the frame dropping interval to obtain the second preview image set at the second frame rate.

[0213] Optionally, in this embodiment, the processor 110 is specifically configured to perform image processing on long-exposure image frames in the first preview image set to obtain and display a first image, wherein the image resolution of the first image is equal to the first frame rate. Alternatively, image processing is performed on long-exposure image frames and short-exposure image frames in the first preview image set to obtain and display a first image, wherein the image resolution of the first image is greater than the first frame rate.

[0214] Optionally, in this embodiment of the application, the processor 110 is specifically configured to process the long exposure image frame and the corresponding short exposure image frame in the first preview image set through an image processor to obtain a first long exposure image frame and a corresponding first short exposure image frame; obtain the tonal information of the first long exposure image frame through a tonal transfer model, and map the tonal information into the first short exposure image frame to obtain a second short exposure image frame; perform image fusion processing on the first long exposure image frame and the second short exposure image frame to obtain and display the first image.

[0215] Optionally, in this embodiment, the processor 110 is further configured to acquire a training image set and a reference image frame, wherein the training image frame set includes long-exposure training image frames and short-exposure training image frames; input the training image set and the reference image frame into an initial model; perform tonal feature extraction processing on the long-exposure training image frames to obtain tonal feature information; map the tonal feature information onto the short-exposure training image frames through the initial model to obtain a third short-exposure image frame; calculate a loss value set based on the third short-exposure image frame and the reference image frame, wherein the loss value set is used to characterize the image texture difference, image brightness difference, and image noise difference between the third short-exposure image frame and the reference image frame; and train the initial model based on the loss value set to obtain a tonal transfer model.

[0216] Optionally, in this embodiment, the aforementioned set of loss values ​​includes a first loss value, a second loss value, and a third loss value. The first loss value is used to characterize the image texture difference between the third short-exposure image frame and the reference image frame; the second loss value is used to characterize the image brightness difference between the third short-exposure image frame and the reference image frame; and the third loss value is used to characterize the image noise difference between the third short-exposure image frame and the reference image frame. Specifically, the processor 110 is used to perform cross-entropy processing on the image texture feature information of the third short-exposure image frame and the image texture feature information of the reference image frame to obtain the first loss value; to perform cross-entropy processing on the image brightness feature information of the third short-exposure image frame and the image brightness feature information of the reference image frame to obtain the second loss value; and to perform cross-entropy processing on the image noise feature information of the third short-exposure image frame and the image noise feature information of the reference image frame to obtain the third loss value.

[0217] In the electronic device provided in this application embodiment, the electronic device can dynamically determine the long exposure duration and short highlight duration corresponding to the current shooting scene according to the current shooting scene, and acquire a set of high frame rate image frames through the long exposure duration and short highlight duration. Then, it displays the first image corresponding to the long exposure image frame, which can ensure the image quality of the preview image. Furthermore, when a shooting command is received, the second image can be obtained by image processing through a set of low frame rate image frames, which can reduce the power consumption of the electronic device when shooting images in high frame rate mode.

[0218] The electronic device provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0219] For details on the beneficial effects of the various implementation methods in this embodiment, please refer to the beneficial effects of the corresponding implementation methods in the above method embodiments. To avoid repetition, these will not be repeated here.

[0220] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0221] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0222] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0223] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0224] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0225] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0226] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0227] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here.

[0228] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0229] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0230] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A shooting method, characterized in that, The method includes: Based on the exposure information corresponding to the first shooting scene, a first preview image set is collected. The exposure information includes a first long exposure duration and a first short exposure duration. The first preview image set includes a long exposure image frame at a first frame rate corresponding to the exposure information and a short exposure image frame corresponding to the long exposure image frame at the first frame rate. The first image is displayed based on the image frames in the first preview image set, and the second preview image set is cached. The second preview image set includes long exposure image frames at the second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate. The second frame rate is less than the first frame rate. Upon receiving a shooting instruction, a second image is generated based on the second set of preview images.

2. The method according to claim 1, characterized in that, Before the step of acquiring the first set of preview images based on the exposure information corresponding to the first shooting scene, the method further includes: When the preview interface of the first shooting mode is displayed, the first shooting scene in which the electronic device is located is determined based on the scene information in the first shooting mode. The scene information includes the motion speed information of the shooting subject and the brightness information of the shooting environment. When the first shooting scenario is a motion shooting scenario, the weight of long exposure time is determined based on the brightness information of the shooting environment, and the weight of short exposure time is determined based on the motion speed information of the subject being shot. The first long exposure duration is determined based on the long exposure duration weight and the preset long exposure duration corresponding to the first shooting mode, and the first short exposure duration is determined based on the short exposure duration weight and the preset short exposure duration corresponding to the first shooting mode.

3. The method according to claim 1 or 2, characterized in that, The step of collecting a first set of preview images based on the exposure information corresponding to the first shooting scene includes: The first exposure time is obtained by summing the first long exposure time and the first short exposure time; When the first exposure duration is less than the first threshold, a long exposure image frame is acquired based on the first long exposure duration, and a short exposure image frame corresponding to the long exposure image frame is acquired based on the first short exposure duration. Wherein, the first threshold is the duration of displaying one frame of image at the first frame rate.

4. The method according to claim 1, characterized in that, Before caching the second set of preview images, the method further includes: The frame dropping interval is determined based on the first frame rate and the second frame rate; Based on the frame dropping interval, the image frames in the first preview image set are processed by dropping frames to obtain the second preview image set at the second frame rate.

5. The method according to claim 1, characterized in that, Displaying the first image based on image frames from the first preview image set includes: Image processing is performed on long-exposure image frames in the first preview image set to obtain and display the first image, wherein the image resolution of the first image is equal to the first frame rate; or, Image processing is performed on the long-exposure image frames and short-exposure image frames in the first preview image set to obtain and display the first image, wherein the image resolution of the first image is greater than the first frame rate.

6. The method according to claim 5, characterized in that, The step of processing the long-exposure image frames and short-exposure image frames in the first preview image set to obtain and display the first image includes: The image processor processes the long exposure image frames and corresponding short exposure image frames in the first preview image set to obtain the first long exposure image frame and the corresponding first short exposure image frame. The tonal information of the first long exposure image frame is obtained by using a tonal transfer model, and the tonal information is mapped onto the first short exposure image frame to obtain the second short exposure image frame. The first long-exposure image frame and the second short-exposure image frame are subjected to image fusion processing to obtain and display the first image.

7. The method according to claim 6, characterized in that, The method further includes: Obtain a set of training images and reference image frames, wherein the set of training image frames includes long-exposure training image frames and short-exposure training image frames; The training image set and the reference image frame are input into the initial model, and the long exposure training image frame is processed to extract tonal features to obtain tonal feature information. Using the initial model, the tonal feature information is mapped into the short exposure training image frame to obtain the third short exposure image frame; Based on the third short-exposure image frame and the reference image frame, a set of loss values ​​is calculated. The set of loss values ​​is used to characterize the differences in image texture, image brightness, and image noise between the third short-exposure image frame and the reference image frame. The initial model is trained based on the set of loss values ​​to obtain the tone transfer model.

8. The method according to claim 7, characterized in that, The set of loss values ​​includes a first loss value, a second loss value, and a third loss value. The first loss value is used to characterize the image texture difference between the third short exposure image frame and the reference image frame. The second loss value is used to characterize the image brightness difference between the third short exposure image frame and the reference image frame. The third loss value is used to characterize the image noise difference between the third short exposure image frame and the reference image frame. The calculation of the loss value set based on the third short-exposure image frame and the reference image frame includes: The image texture feature information of the third short-exposure image frame and the image texture feature information of the reference image frame are subjected to cross-entropy processing to obtain the first loss value; The image brightness feature information of the third short-exposure image frame and the image brightness feature information of the reference image frame are subjected to cross-entropy processing to obtain the second loss value; The image noise feature information of the third short-exposure image frame and the image noise feature information of the reference image frame are subjected to cross-entropy processing to obtain the third loss value.

9. A shooting device, characterized in that, The imaging device includes: an acquisition module and a processing module; The acquisition module is used to acquire a first set of preview images based on the exposure information corresponding to the first shooting scene. The exposure information includes a first long exposure duration and a first short exposure duration. The first set of preview images includes long exposure image frames at a first frame rate corresponding to the exposure information and short exposure image frames corresponding to the long exposure image frames at the first frame rate. The processing module is configured to display a first image based on image frames in the first preview image set, and cache a second preview image set, the second preview image set including long exposure image frames at a second frame rate and short exposure image frames corresponding to the long exposure image frames at the second frame rate, the second frame rate being less than the first frame rate; and generate a second image based on the second preview image set upon receiving a shooting instruction.

Images