Shooting processing method and related equipment

Abstract

The embodiment of the invention provides a shooting processing method and related equipment, and the method comprises the steps: responding to an operation of starting a camera application program by a user, and collecting an image flow through a camera; in response to a shooting operation of a user, generating a first image sequence according to the image stream; acquiring a plurality of image frames from the first image sequence to generate a second image sequence; selecting a reference image frame from the second image sequence based on a score for each image frame in the second image sequence; obtaining a plurality of image frames from the first image sequence according to the reference image frame, and generating a third image sequence; generating a target video according to the plurality of image frames in the third image sequence; obtaining a plurality of image frames from the first image sequence according to the reference image frame, and generating a fourth image sequence; and generating a target image according to the plurality of image frames in the fourth image sequence. According to the embodiment of the invention, the shooting operation can be responded, and the precise wonderful action picture and the wonderful moment video can be automatically generated according to the image stream collected by the camera.

Description

Technical Field

[0001] The present application relates to the field of smart terminal technology, and in particular to a shooting processing method and related equipment. Background Art

[0002] With the development of terminal technology, smart terminal devices such as smartphones and tablets all have shooting functions, allowing users to record various scenes by taking photos or videos. When shooting sports scenes, users often need to capture the wonderful movements during the movement. However, the wonderful movements are fleeting, and there is a delay between the user seeing the wonderful movements and executing the shooting operation. Therefore, the photos of the wonderful movements obtained by taking photos cannot meet the needs of users, and the videos of the wonderful movements obtained by taking videos have low resolution and poor image quality. Summary of the invention

[0003] In view of the above, it is necessary to provide a shooting and processing method and related equipment to solve the problem that the photos of wonderful actions obtained by taking photos cannot meet user needs, and the video quality of wonderful moments obtained by taking videos is poor.

[0004] In a first aspect, the present application provides a shooting processing method, which is applied to an electronic device, and the method includes: responding to a user's operation of opening a camera application, collecting an image stream through a camera; responding to the user's shooting operation, generating a first image sequence according to the image stream; acquiring multiple image frames from the first image sequence to generate a second image sequence; based on a score for each image frame in the second image sequence, selecting a reference image frame from the second image sequence; acquiring multiple image frames from the first image sequence according to the reference image frame to generate a third image sequence; generating a target video according to the multiple image frames in the third image sequence; acquiring multiple image frames from the first image sequence according to the reference image frame to generate a fourth image sequence; and generating a target image according to the multiple image frames in the fourth image sequence.

[0005] Based on the above technical solution, in the process of shooting sports scenes in response to the user's shooting operation, high-quality and high-precision target photos, that is, wonderful action photos, can be automatically generated according to the image stream collected by the camera, and target videos, that is, videos of wonderful actions, can be automatically generated. The video can record dynamic wonderful actions, and the video length is also shorter, which reduces the storage space occupied by electronic devices.

[0006] In one possible implementation, the method of responding to a user's shooting operation and generating a first image sequence according to the image stream includes: according to the moment when the user performs the shooting operation, obtaining from the image stream image frames captured by the camera within a first preset time length before the moment and image frames captured by the camera within a second preset time length after the moment, to generate the first image sequence.

[0007] Based on the above technical solution, by collecting multiple image frames as candidate images for generating wonderful moment videos and wonderful action images, the accuracy of the generated wonderful moment videos and wonderful action images can be improved.

[0008] In a possible implementation, acquiring multiple image frames from the first image sequence to generate the second image sequence includes: acquiring one image frame from every first preset number of consecutive image frames in the first image sequence to generate the second image sequence.

[0009] Based on the above technical solution, a small number of image frames are selected from multiple image frames by frame extraction for subsequent image optimization, thereby improving the efficiency of image scoring and optimization.

[0010] In one possible implementation, the selecting a reference image frame from the second image sequence based on the score of each image frame in the second image sequence includes: inputting each image frame in the second image sequence into at least one scoring model to obtain at least one score for each image frame; and selecting the reference image frame from the second image sequence based on the at least one score for each image frame.

[0011] Based on the above technical solution, at least one scoring model can be used to accurately screen out reference image frames with higher scores from multiple image frames. Based on the reference image frames with higher scores, high-precision wonderful moment videos and wonderful action images can be generated, and the generation efficiency of wonderful moment videos and wonderful action images can also be improved.

[0012] In a possible implementation, the at least one scoring model includes an action scoring model, an image clarity scoring model, and an expression scoring model.

[0013] Based on the above technical solution, multiple image frames can be scored from multiple dimensions such as action, image quality, expression, etc., so as to accurately screen out reference image frames.

[0014] In a possible implementation manner, the method further includes: identifying a target object in a plurality of image frames in the first image sequence; and calculating a motion amplitude of the target object according to the plurality of image frames in the first image sequence.

[0015] Based on the above technical solution, motion detection can be performed on multiple image frames in an image sequence, which is convenient for subsequent determination of the number of images and frame rate during video interpolation, so that the generated wonderful moment video can reflect the motion amplitude of the target object.

[0016] In one possible implementation, identifying the target object in the multiple image frames of the first image sequence includes: inputting the multiple image frames of the first image sequence into a target detection model, and outputting the target objects in the multiple image frames of the first image sequence through the target detection model.

[0017] Based on the above technical solution, the target object in the image frame can be accurately identified through the target detection model, so as to perform motion detection on the target object and determine the motion amplitude of the target object.

[0018] In a possible implementation, calculating the motion amplitude of the target object according to the plurality of image frames in the first image sequence includes: calculating the motion amplitude of the target object by using an optical flow method according to every two adjacent image frames in the first image sequence.

[0019] Based on the above technical solution, the optical flow method can be used to accurately perform motion detection on multiple image frames in an image sequence.

[0020] In a possible implementation, the step of acquiring multiple image frames from the first image sequence according to the reference image frame to generate a third image sequence includes: if the motion amplitude of the target object is greater than or equal to a preset motion amplitude threshold, acquiring multiple image frames within a third preset time length from the first image sequence according to the reference image frame to generate the third image sequence; or if the motion amplitude of the target object is less than the preset motion amplitude threshold, acquiring multiple image frames within a fourth preset time length from the first image sequence according to the reference image frame to generate the third image sequence, the third preset time length being less than the fourth preset time length.

[0021] Based on the above technical solution, different numbers of image frames can be selected according to the motion amplitude of the target object to generate a video of the wonderful moment, thereby improving the generation efficiency of the wonderful moment video, so that the wonderful moment video can reflect the motion amplitude of the target object.

[0022] In a possible implementation, the step of acquiring multiple image frames within a third preset time length from the first image sequence based on the reference image frame to generate the third image sequence includes: acquiring multiple image frames within a fifth preset time length before the reference image frame from the first image sequence, and acquiring multiple image frames within a sixth preset time length after the reference image frame to generate the third image sequence, wherein the sum of the fifth preset time length and the sixth preset time length is the third preset time length.

[0023] Based on the above technical solution, a plurality of image frames can be selected from before and after the reference image frame to generate a wonderful moment video, so that the generated wonderful moment video is more accurate.

[0024] In a possible implementation, generating a target video based on multiple image frames in the third image sequence includes: if the motion amplitude of the target object is greater than or equal to the preset motion amplitude threshold, interpolating the multiple image frames in the third image sequence according to a first frame rate to generate the target video of a ninth preset duration; or if the motion amplitude of the target object is less than the preset motion amplitude threshold, interpolating the multiple image frames in the third image sequence according to a second frame rate to generate the target video of a ninth preset duration, wherein the first frame rate is greater than the second frame rate.

[0025] Based on the above technical solution, video interpolation processing can be performed at different frame rates according to the motion amplitude of the target object, so that the video of the wonderful moment can reflect the motion amplitude of the target object.

[0026] In a possible implementation, the step of acquiring multiple image frames within a fourth preset time length from the first image sequence based on the reference image frame to generate the third image sequence includes: acquiring multiple image frames within a seventh preset time length before the reference image frame from the first image sequence, and acquiring multiple image frames within an eighth preset time length after the reference image frame to generate the third image sequence, wherein the sum of the seventh preset time length and the eighth preset time length is the fourth preset time length.

[0027] Based on the above technical solution, a plurality of image frames can be selected from before and after the reference image frame to generate a wonderful moment video, so that the generated wonderful moment video is more accurate.

[0028] In a possible implementation, the method further includes: storing the target video in a gallery application of the electronic device; optimizing the reference image frame, and using the optimized reference image frame as a cover image of the target video.

[0029] Based on the above technical solution, by setting the cover image of the target video, it is convenient for users to view the wonderful moment video.

[0030] In a possible implementation, acquiring multiple image frames from the first image sequence according to the reference image frame to generate a fourth image sequence includes: acquiring the reference image frame and a second preset number of image frames before the reference image frame from the first image sequence to generate the fourth image sequence.

[0031] Based on the above technical solution, multiple image frames can be screened out as candidate images for generating wonderful action images, thereby improving the accuracy of the generated wonderful action images.

[0032] In a possible implementation, generating a target image based on multiple image frames in the fourth image sequence includes: fusing multiple image frames in the fourth image sequence to obtain a fused image; and optimizing the fused image to obtain the target image.

[0033] Based on the above technical solution, multiple image frames can be fused and optimized to generate high-quality exciting action images.

[0034] In a possible implementation manner, the image stream includes a preview image stream and a photo image stream, and a resolution of image frames in the photo image stream is greater than a resolution of image frames in the preview image stream.

[0035] Based on the above technical solution, image motion detection and video generation can be performed based on the preview image stream, thereby improving the efficiency of generating wonderful moment videos and generating high-quality wonderful action images based on the photographed image stream.

[0036] In a second aspect, the present application provides an electronic device, comprising a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device executes the above-mentioned shooting processing method.

[0037] In a third aspect, the present application provides a chip coupled to a memory in an electronic device, wherein the chip is used to control a processor of the electronic device to execute the above-mentioned shooting processing method.

[0038] In a fourth aspect, the present application provides a computer storage medium storing program instructions, which, when executed on an electronic device, enables a processor of the electronic device to execute the above-mentioned shooting processing method.

[0039] In addition, the technical effects brought about by the second to fourth aspects can be found in the descriptions related to the methods of each design in the above method part, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the interface for an electronic device to take photos while recording.

[0041] Figure 2 It is a schematic diagram of the interface for an electronic device to record multiple results at once.

[0042] Figure 3 It is a schematic diagram of the interface of an electronic device performing Hawkeye capture.

[0043] Figure 4 It is a schematic diagram of the interface for an electronic device to take photos and dynamic images at the same time.

[0044] Figure 5 It is a software architecture diagram of an electronic device provided in one embodiment of the present application.

[0045] Figure 6 It is a flowchart of a shooting processing method provided in one embodiment of the present application.

[0046] Figure 7 This is a software architecture diagram of a camera service of an electronic device provided in one embodiment of the present application.

[0047] Figure 8 It is a schematic diagram of the preview stream and the photo stream provided in one embodiment of the present application.

[0048] Fig. 9 It is a schematic diagram of a camera application program interface provided in one embodiment of the present application.

[0049] Fig.10 It is a schematic diagram of generating a third image sequence provided by an embodiment of the present application.

[0050] Fig.11 It is a schematic diagram of the interface of the gallery application provided in one embodiment of the present application.

[0051] Fig.12 This is a flowchart of selecting a reference image frame from a third image sequence provided by an embodiment of the present application.

[0052] Fig.13 This is a flowchart of generating a fourth image sequence provided by an embodiment of the present application.

[0053] Fig.14 It is a schematic diagram of generating a fourth image sequence provided by an embodiment of the present application.

[0054] Fig.15 This is a flowchart for generating a target video provided by an embodiment of the present application.

[0055] Fig.16 It is a schematic diagram of multiple image frames in a fourth image sequence provided by an embodiment of the present application.

[0056] Fig.17 It is a schematic diagram of multiple image frames after interpolation processing provided by an embodiment of the present application.

[0057] Fig.18 It is a schematic diagram of generating a fifth image sequence provided by an embodiment of the present application.

[0058] Fig.19 This is a flowchart for generating a target image provided by an embodiment of the present application.

[0059] Fig. 20 This is a hardware architecture diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] The terms "first" and "second" involved in the embodiments of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" refers to one or more. "Multiple" refers to two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c. In the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0062] With the development of terminal technology, smart terminal devices such as smartphones and tablets all have shooting functions, allowing users to record various scenes by taking photos or videos. When shooting sports scenes, users often need to capture the wonderful movements during the movement. However, the wonderful movements are fleeting, and there is a delay between the user seeing the wonderful movements and executing the shooting operation. Therefore, the photos of the wonderful movements obtained by taking photos cannot meet the needs of users, and the videos of the wonderful movements obtained by taking videos have low resolution and poor image quality. It can be seen that in the process of shooting sports scenes, how to accurately capture and record the wonderful movements during the movement through terminal devices is an urgent problem to be solved in the industry.

[0063] See also Figure 1 As shown, it is a schematic diagram of the interface for an electronic device to take photos during recording. An electronic device may provide a function of taking photos during recording. For example, the electronic device provides a photo control in the recording interface. When shooting a sports scene, the user can record the sports process through the electronic device, and when a wonderful action is found while recording, the photo control is manually triggered to take photos of the wonderful action, thereby recording the video and photos of the wonderful action at the same time. However, the video recorded in this way is long, which easily occupies the storage space of the electronic device, and the video resolution is low and the image quality is poor. In addition, the photos of the wonderful action are taken manually by the user. Since there is a delay between the user seeing the wonderful action and executing the shooting operation to take the photo, the photos actually taken are also delayed, and the moment of the wonderful action is often missed, resulting in the acquisition of photos that are not wonderful actions.

[0064] See also Figure 2 As shown, it is a schematic diagram of the interface of an electronic device for recording multiple results at once. The electronic device can also provide a function of recording multiple results at once. For example, when shooting a sports scene, the user can record the sports process through the electronic device. After the recording is completed, the electronic device can automatically generate photos of wonderful actions based on the captured video frames. However, the video recorded in this way is long, which is easy to occupy the storage space of the electronic device, and the video resolution is low and the image quality is poor. In addition, the photos of wonderful actions are generated based on the video frames, and the photo resolution is low and the image quality is also poor.

[0065] See also Figure 3The figure shows an interface diagram of an electronic device for Hawkeye capture. The electronic device can also provide a Hawkeye capture function. For example, when shooting a sports scene, the electronic device can identify a specified wonderful action in the sports scene based on the preview image frame, and automatically take a photo of the wonderful action when the specified wonderful action is identified. However, this method can only record photos of wonderful actions, but not videos of wonderful actions, and often can only identify specified wonderful actions, resulting in limited types of wonderful actions that can be captured.

[0066] See also Figure 4 As shown, it is a schematic diagram of the interface of an electronic device taking photos and dynamic images at the same time. The electronic device can also provide the function of recording dynamic images while taking photos. For example, when shooting a sports scene, the user can manually take photos of the wonderful action through the electronic device when discovering the wonderful action, and the electronic device simultaneously generates a dynamic image of the wonderful action based on the preview image frame. However, the dynamic image recorded in this way has a low resolution and poor image quality. In addition, the photo of the wonderful action is taken manually by the user. Since there is a delay between the user seeing the wonderful action and executing the shooting operation to take the photo, the photo actually taken also has a delay and may not be a photo of the wonderful action, and the action coherence in the dynamic image generated thereby may be poor.

[0067] In order to solve the problem that the accuracy of photos of wonderful actions obtained by taking photos is low and cannot meet user needs, and the video quality of wonderful moments obtained by taking videos is poor, the embodiment of the present application provides a shooting processing method. In the process of shooting a motion scene in response to the user's shooting operation, high-quality and high-precision photos of wonderful actions can be automatically generated according to the image stream collected by the camera, and videos of wonderful actions can be automatically generated. The video can record dynamic wonderful actions, and the video length is also shorter, which reduces the storage space occupied by electronic devices.

[0068] See also Figure 5 As shown, it is a software architecture diagram of the electronic device provided in the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the Android system is divided into four layers, from top to bottom, respectively, the application layer 101, the framework layer 102, the Android runtime (Androidruntime) and the system library 103, the hardware abstraction layer 104, the kernel layer 105, and the hardware layer 106.

[0069] The application layer 101 may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, device control service and other applications.

[0070] The framework layer 102 provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0071] Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window, such as a text message in the status bar, a beep, a vibration of an electronic device, a flashing indicator light, etc.

[0072] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0073] The application layer 101 and the framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and the framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0074] The system library 103 may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (eg, OpenGL ES), a 2D graphics engine (eg, SGL), and the like.

[0075] Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0076] The hardware abstraction layer 104 runs in the user space, encapsulates the kernel layer driver, and provides a calling interface to the upper layer.

[0077] The kernel layer 105 is a layer between hardware and software and includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0078] The kernel layer 105 is the core of the operating system of the electronic device. It is the first layer of software expansion based on hardware, provides the most basic functions of the operating system, is the basis of the operating system, is responsible for managing the system's processes, memory, device drivers, files and network systems, and determines the performance and stability of the system. For example, the kernel can determine the operating time of an application on a certain part of the hardware.

[0079] The kernel layer 105 includes programs closely related to the hardware, such as interrupt handlers, device drivers, etc., and also includes basic, common, and high-frequency modules, such as clock management modules, process scheduling modules, etc., and also includes key data structures. The kernel layer can be set in the processor or solidified in the internal memory.

[0080] The hardware layer 106 includes the hardware of the electronic device, such as a display screen, buttons, a camera, etc.

[0081] See also Figure 6 FIG. 1 is a flowchart of a shooting processing method provided by an embodiment of the present application. The method is applied in an electronic device, and the shooting processing method includes:

[0082] S101, in response to a user's operation of starting a camera application, an image stream is collected through a camera.

[0083] In one embodiment of the present application, the user can click on the camera application icon on the main interface of the electronic device, or click on the camera start control on the lock screen interface of the electronic device, or can open the camera application by touching or pressing a preset physical button. In response to the user's operation of opening the camera application, the electronic device captures an image of the scene in front of it through the camera according to a preset sampling frequency to collect an image stream. For example, the sampling frequency can be 30Hz, 32Hz or 60Hz.

[0084] See also Figure 7 The figure shows the software architecture diagram of the camera service of the electronic device provided by the embodiment of the present application. In response to the user's operation of opening the camera application, the camera application creates a camera process, instantiates the camera service (CameraService), the cache queue (CameraBufferQuene) corresponding to the camera application, the cache read service (BlastBufferQueue, BBQ), the synthesis service (SurfaceFlinger), the camera provider (CameraProvider), the hardware synthesizer (Hardware Composer, HWC), the signal center (DispSync), etc., and creates a layer of controls to be displayed in the camera process, for example Figure 7 The layers of control 1, control 2, control 3, control 4, etc. Among them, control 1, control 2, control 3, and control 4 can be a shooting mode selection control, a shooting control, a control for setting shooting parameters, a control for opening a gallery application, and a lens switching control.

[0085] In one embodiment of the present application, HWC periodically generates a VSync-HW signal according to the refresh frequency of the display screen, and DispSync assigns a corresponding VSync-APP signal to the user interface (UI) thread corresponding to the camera application according to the periodically generated VSync-HW signal, and periodically sends the generated VSync-APP signal to the UI thread of the camera application, so that the UI thread triggers the drawing and rendering processing after receiving the VSync-APP signal.

[0086] In one embodiment of the present application, DispSync also allocates a corresponding VSync-SF signal to SurfaceFlinger based on the periodically generated VSync-HW signal, and periodically sends the generated VSync-SF signal to SurfaceFlinger, so that after receiving the VSync-SF signal, SurfaceFlinger obtains image frames from CameraBufferQuene through BBQ, and then performs synthesis processing on the obtained image frames. Among them, the image frames cached in CameraBufferQuene can be image frames obtained by CameraProvider processing the image frames in the original image stream captured by the camera. The acquisition of the original image stream can be obtained by the camera continuously acquiring the preview shooting request or shooting request generated by the camera application. Among them, the generation of the preview shooting request or shooting request can be generated by the camera application according to the current shooting mode after the camera application is started.

[0087] In one embodiment of the present application, after CameraProvider receives the preview shooting request or shooting request transmitted by CameraService, it transmits the preview shooting request or shooting request to the camera driver, and the camera driver sends the preview shooting request or shooting request to the camera, and drives the camera to capture image frames according to a preset sampling frequency.

[0088] In one embodiment of the present application, after the camera captures the image frame, it continuously reports the image frame in the original image stream to the camera driver, and then the camera driver transmits the image frame to the CameraProvider, which processes the image frame accordingly. The CameraProvider transmits the processed image frame to the CameraService, and the CameraService adds the image frame to the CameraBufferQuene.

[0089] In one embodiment of the present application, SurfaceFlinger performs synthesis processing on the image frames cached in CameraBufferQuene, which is triggered by the VSync-SF signal periodically generated by DispSync. After receiving the VSync-SF signal, SurfaceFlinger obtains the cached image frames from CameraBufferQuene through BBQ, and BBQ also synchronizes the timestamp of each acquired image frame added to CameraBufferQuene to SurfaceFlinger.

[0090] If SurfaceFlinger obtains the cached image frame from CameraBufferQuene through BBQ, it will synthesize the image frame to obtain the synthesized image corresponding to the image frame, and transmit the synthesized image to HWC. At the same time, SurfaceFlinger will send an instruction to destroy the image frame to BBQ, and BBQ will transmit the destruction instruction to CameraBufferQuene. CameraBufferQuene notifies CameraService to call the dequeue function to delete the image frame cached in CameraBufferQuene.

[0091] The dequeue function is used to remove the first function in the specified queue of each matching element and execute the removed function, such as deleting the image frame at the head of the queue. When SurfaceFlinger obtains the cached image frame from CameraBufferQuene through BBQ, it can obtain it from the head of the queue. Therefore, after synthesizing the image, calling the dequeue function in the above way can delete the used image frame from CameraBufferQuene.

[0092] HWC transmits the composite image sent by SurfaceFlinger to the display driver, and the display driver drives the display screen to display the screen corresponding to the composite image.

[0093] In one embodiment of the present application, the image stream includes a preview stream and a photo stream, the preview stream includes a plurality of preview image frames, the photo stream includes a plurality of photo image frames, and the resolution of the photo image frame is greater than the preview image frame. For example, the resolution of the photo image frame is 8000*6000, and the resolution of the preview image frame is 4000*3000. After the camera application is turned on, the camera service (CameraService) of the framework layer and the camera provider (CameraProvider) of the hardware abstraction layer are initialized. After the initialization is completed, the camera application sends a preview shooting request to the camera service, the camera service sends a preview shooting request to the camera provider, and the camera provider sends a preview shooting request to the camera driver of the camera of the kernel layer. The camera driver can drive the camera to collect image frames, and return the collected multiple image frames to the camera provider. The camera provider returns the multiple image frames to the camera service, and the camera service sends the multiple image frames to the drawing and rendering service of the framework layer, and the multiple image frames are sent and displayed by the drawing and rendering service to generate a preview stream of the camera application. The camera service also stores the multiple image frames in the buffer to generate a photo stream of the camera application.

[0094] In another embodiment of the present application, the image stream may include a photo stream. In this case, the image frames cached and displayed by the camera application are all photo image frames in the photo stream.

[0095] S102, in response to a shooting operation of a user, generating a first image sequence according to an image stream.

[0096] In one embodiment of the present application, after the camera application is opened, the user can perform a shooting operation by clicking the shooting control on the camera application interface. The camera application responds to the user's shooting operation, generates and sends a shooting request to the camera service, and the camera service sends the shooting request to the camera provider. The camera provider sends the shooting request to the camera driver of the camera, drives the camera to capture image frames through the camera driver, and returns the captured image frames to the camera provider. The camera provider returns the image frames to the camera service, and the camera service sends the image frames to the gallery application for storage. Then, the camera application continues to send preview shooting requests, the camera continues to capture image frames, and generates the preview stream and photo stream of the camera application based on the captured image frames.

[0097] See also Figure 8 As shown, it is a schematic diagram of a preview stream and a photo stream provided by an embodiment of the present application. In an embodiment of the present application, the preview stream and the photo stream of the camera application can be stored in a cache, for example, in a rotating cache, and the rotating cache has a preset capacity, and the preset capacity can be the number of image frames, for example, the preset capacity is 60, 70, 80 or other values. The rotating buffer is a data structure for storing and managing data in a first-in-first-out order. The rotating buffer is usually a fixed-size circular queue composed of a series of fixed-size buffers. When data is written to the rotating buffer, it is written to the currently active buffer in order and gradually advances with the write position of the data. When the buffer is full, the oldest data will be overwritten or deleted to make room for new data. In this way, the rotating cache caches the image frames according to the first-in-first-out principle. For example, if the number of images cached in the rotating cache exceeds the preset capacity, the image frame with the earliest cache time is deleted.

[0098] In one embodiment of the present application, an image frame captured in response to a capture operation, a first preset number of image frames before the image frame captured in response to the capture operation, and a second preset number of image frames after the image frame captured in response to the capture operation can be obtained from the image stream to generate a first image sequence. For example, Figure 8The image frame t0 in is an image frame obtained in response to the shooting operation. The sum of the first preset number and the second preset number is the preset capacity of the round-robin cache. For example, the preset capacity of the round-robin cache is 60, the first preset number is 50, and the second preset number is 10. The image streams stored in the round-robin cache form an image sequence. For example, the preview stream stored in one round-robin cache forms a first image sequence, and the photo stream stored in another round-robin cache forms a second image sequence.

[0099] In another embodiment of the present application, if the image stream includes a preview stream and a photo stream, in response to the user's shooting operation, the time T when the user performs the shooting operation is determined, and according to the time T when the user performs the shooting operation, the preview image frames collected by the camera within the first preset time length before the time T, the preview image frames corresponding to the time T, and the preview image frames collected by the camera within the second preset time length after the time T are obtained from the preview stream to generate a first image sequence. For example, Figure 8 The image frame t0 in the preview stream shown is the preview image frame corresponding to the time T. The sum of the first preset duration and the second preset duration can be set as the preset duration threshold. For example, the preset duration threshold is 4 / 3 seconds, the first preset duration is 1 second, and the second preset duration is 1 / 3 seconds. In other embodiments, the preset duration threshold can also be other durations, for example, the preset duration threshold is determined based on the first preset duration, the second preset duration, and a fixed duration.

[0100] In another embodiment of the present application, according to the time T when the user performs a shooting operation, the camera captures the image frames within a first preset time period before the time T, the image frames corresponding to the time T, and the image frames captured by the camera within a second preset time period after the time T from the photo stream to generate a second image sequence. For example, Figure 8 The image frame t0 in the photo stream shown is the photo image frame corresponding to time T.

[0101] In another embodiment of the present application, if the image stream includes a photo stream, in response to the user's shooting operation, the moment T when the user performs the shooting operation is obtained, and based on the moment T when the user performs the shooting operation, the photo image frames captured by the camera within a first preset time length before the moment T, the photo image frames corresponding to the moment T, and the photo image frames captured by the camera within a second preset time length after the moment T are obtained from the photo stream to generate a first image sequence.

[0102] In one embodiment of the present application, when a user uses an electronic device to shoot a sports scene, the user can perform a shooting operation when the target in the sports scene is considered to perform an exciting action, so that the electronic device responds to the shooting operation and generates a first image sequence corresponding to the preview stream and a second image sequence corresponding to the photo stream according to the image stream. Fig. 9As shown, it is a schematic diagram of a camera application program interface provided by an embodiment of the present application. For example, a user uses an electronic device to shoot a scene of an athlete long jumping, and can trigger a shooting control A when the athlete jumps to the highest point to perform a shooting operation. The electronic device responds to the shooting operation and generates a first image sequence and a second image sequence according to the image stream.

[0103] S103, acquiring multiple image frames from the first image sequence to generate a third image sequence.

[0104] See also Fig.10 As shown, it is a schematic diagram of generating a third image sequence provided by an embodiment of the present application. In one embodiment of the present application, one image frame is obtained from every third consecutive preset number of image frames in the first image sequence (for example, the first image sequence corresponding to the preview stream) to generate a second image sequence. For example, the third preset number is 2, 3, 5 or other values. Among them, taking the image frame with the earliest shooting moment in the first image sequence as a reference, one image frame is obtained from every third consecutive preset number of image frames to generate a third image sequence. Taking the third preset number as 3 as an example, one image frame is obtained from every 3 image frames in the first image sequence. Assuming that a total of 30 image frames are obtained, the third image sequence is generated based on the 30 image frames obtained.

[0105] S104: Select a reference image frame from the third image sequence based on the score of each image frame in the third image sequence.

[0106] In one embodiment of the present application, each image frame in the third image sequence is input into at least one preset scoring model to obtain at least one score for each image frame, and a reference image frame is selected from the third image sequence based on the at least one score for each image frame. The at least one scoring model includes but is not limited to: an action scoring model, an image clarity scoring model, and an expression scoring model.

[0107] In one embodiment of the present application, if the number of at least one scoring model is one, each image frame in the third image sequence is input into the scoring model to obtain the score of each image frame, and the image frame with the highest score is selected from the third image sequence as the reference image frame. If the number of at least one scoring model is multiple, each image frame in the third image sequence is input into multiple scoring models respectively to obtain the score of each image frame corresponding to each scoring model, thereby obtaining multiple scores for each image frame, and the comprehensive score of each image frame is determined by summing or weighted summing the multiple scores, and the image frame with the highest comprehensive score is selected from the third image sequence as the reference image frame.

[0108] In one embodiment of the present application, by selecting an image frame with the highest comprehensive score from the third image sequence as a reference image frame, a video of a wonderful moment in a captured sports scene and an image of a wonderful action can be accurately generated based on the reference image frame.

[0109] S105 , acquiring multiple image frames from the first image sequence according to the reference image frame to generate a fourth image sequence.

[0110] In one embodiment of the present application, a target object in multiple image frames of a first image sequence is identified, motion detection is performed on the target object in the multiple image frames, the motion amplitude of the target object is calculated based on the multiple image frames in the first image sequence, and based on the motion amplitude of the target object and the reference image frames, multiple image frames are obtained from the first image sequence to generate a fourth image sequence.

[0111] S106: Generate a target video according to a plurality of image frames in a fourth image sequence.

[0112] In one embodiment of the present application, a target video is generated according to the motion amplitude of the target object and a plurality of image frames in the fourth image sequence, wherein the target video is a video of wonderful actions in the shooting scene, and the video may be a slow motion video or a time-lapse video.

[0113] S107, acquiring multiple image frames from the second image sequence according to the reference image frame to generate a fifth image sequence.

[0114] In one embodiment of the present application, a reference image frame and a fourth preset number of image frames before the reference image frame are obtained from a second image sequence corresponding to the photographing stream to generate a fifth image sequence.

[0115] S108, generating a target image according to a plurality of image frames in the fifth image sequence.

[0116] In one embodiment of the present application, multiple image frames in the fifth image sequence are fused to obtain a fused image, and the fused image is optimized using a preset photography algorithm to obtain a target image, wherein the target image is an image of a wonderful action in the shooting scene.

[0117] In another embodiment of the present application, multiple image frames in the fifth image sequence may be optimized first, and then the optimized images may be fused to obtain the target image.

[0118] In the above-mentioned embodiment of the present application, when the user opens the camera application to shoot the sports scene, the camera continuously collects the image stream and responds to the user's shooting operation to automatically generate videos and images of the wonderful actions in the sports scene. The wonderful action images have high image quality and accuracy, the video can record dynamic wonderful actions, and the video length is also short, which reduces the storage space occupied by the electronic device.

[0119] In one embodiment of the present application, the method further includes: storing the target video in a gallery application of an electronic device; optimizing the reference image frame, and using the optimized reference image frame as a cover image of the target video. In a thumbnail interface provided by the gallery application, the target video with the reference image frame as the cover image is displayed. In one embodiment of the present application, the optimization processing of the reference image frame includes but is not limited to noise reduction processing and tone mapping processing, and the tone mapping processing refers to converting a standard dynamic range image into a high dynamic range image.

[0120] In one embodiment of the present application, the method further includes: storing the target image in a gallery application of the electronic device. Fig.11 As shown, it is a schematic diagram of the interface of the gallery application provided by an embodiment of the present application. When a user performs a shooting operation in the camera application, the electronic device generates a video and an image of the wonderful action in the shooting scene in response to the shooting operation, and stores them in the gallery application. The user can view the video of the wonderful action by clicking on the video cover of the wonderful action, or view the image of the wonderful action by clicking on the thumbnail of the image of the wonderful action.

[0121] See also Fig.12 As shown, it is a flowchart of selecting a reference image frame from a third image sequence provided by an embodiment of the present application.

[0122] S1041, input each image frame in the third image sequence into an action scoring model, and output an action score for each image frame through the action scoring model.

[0123] In one embodiment of the present application, the action scoring model can be a convolutional neural network model, and the action scoring model uses multiple wonderful action images under multiple sports scenes as training data for training generation. Wherein, multiple sports scenes include but are not limited to playing basketball, playing football, playing badminton, playing table tennis, running, and jumping. The wonderful action when playing basketball includes but is not limited to shooting action, the wonderful action when playing football includes but is not limited to shooting action, the wonderful action when playing badminton includes but is not limited to spiking action, the wonderful action when playing table tennis includes but is not limited to serving action, and the wonderful action when running includes but is not limited to sprinting action. The wonderful actions under the above sports scenes and sports scenes are only used for illustration, and may not be limited to the above examples in actual application, and the embodiments of the present application are not limited to this. For example, the action scoring model can obtain action scoring results based on these similarities by extracting key point features in the image frame and calculating the similarity between different actions and a single image frame.

[0124] In one embodiment of the present application, each image frame in the third image sequence is input into an action scoring model, features of the image frame are extracted through the action scoring model, whether the image frame contains wonderful action is analyzed based on the extracted features, a determination result of whether the image frame contains wonderful action and a probability of containing wonderful action in the image frame are output, and the probability of containing wonderful action in the image frame is used as the action score of the image frame.

[0125] S1042, input each image frame in the third image sequence into an image clarity scoring model, and output a clarity score of each image frame through the image clarity scoring model.

[0126] In one embodiment of the present application, the image clarity scoring model may be a Brenner gradient function, and the clarity factor of the image frame is calculated by the image clarity scoring model, and the clarity factor is used as the clarity score of the image frame. The calculation formula of the Brenner gradient function is:

[0127] D(f)=∑ y ∑ x |f(x+2,y)-f(x,y)| 2 (1).

[0128] In calculation formula (1), (x, y) is the pixel value of each pixel in the image frame, f(x, y) is the grayscale value of each pixel, and D(f) is the clarity factor of the image frame.

[0129] In other embodiments of the present application, the image clarity scoring model may also be a Tenengrad gradient function, a Laplace operator gradient function, a grayscale variance function, a grayscale variance product function, an energy gradient function, etc.

[0130] In another embodiment of the present application, the target object of each image frame in the third image sequence may also be identified, and a clarity score may be performed on the target object to obtain a clarity score of the image frame, thereby performing a local blur evaluation on the image frame.

[0131] S1043, input each image frame in the third image sequence into an expression scoring model, and output an expression score of each image frame through the expression scoring model.

[0132] In one embodiment of the present application, the expression scoring model may be a convolutional neural network model, and the expression scoring model uses facial images corresponding to multiple preset expressions as training data for training generation. The multiple preset expressions include but are not limited to smiles, tears, excitement, and tension. The above preset expressions are only used for illustration, and may not be limited to the above examples in actual application, and the present application embodiment does not limit this.

[0133] In one embodiment of the present application, each image frame in the third image sequence is input into an expression scoring model, features of the image frame are extracted through the expression scoring model, whether the image frame contains a preset expression is analyzed based on the extracted features, a determination result of whether the image frame contains a preset expression and a probability of containing the preset expression in the image frame are output, and the probability of containing the preset expression in the image frame is used as the expression score of the image frame.

[0134] S1044, determining a comprehensive score for each image frame based on the action score, image clarity score, and expression score of each image frame.

[0135] In one embodiment of the present application, the sum of the action score, image clarity score and expression score of each image frame is calculated to obtain a comprehensive score for each image frame.

[0136] In another embodiment of the present application, the action score, image clarity score and expression score of each image frame are weighted and summed to obtain a comprehensive score of each image frame, wherein the sum of the weight values ​​of the action score, image clarity score and expression score is 1.

[0137] S1045: Determine the image frame with the highest comprehensive score in the third image sequence as the reference image frame.

[0138] In one embodiment of the present application, the comprehensive scores of all image frames in the third image sequence are compared, and the image frame with the highest comprehensive score is determined as the reference image frame.

[0139] The above-mentioned embodiments of the present application can select multiple image frames in the image stream through multiple scoring models, thereby screening out image frames containing wonderful actions as reference image frames, so as to facilitate the generation of accurate videos and images of wonderful actions based on the reference image frames.

[0140] See also Fig.13 As shown, it is a flowchart of generating a fourth image sequence provided by an embodiment of the present application.

[0141] S1051, identifying a target object in a plurality of image frames of a first image sequence.

[0142] In one embodiment of the present application, multiple image frames of the first image sequence are input into a target detection model, and the target objects in the multiple image frames of the first image sequence are determined by the target detection model. The target detection model may be a model established according to the principle of instance segmentation algorithm.

[0143] S1052: Calculate a motion amplitude of the target object according to a plurality of image frames in the first image sequence.

[0144] In one embodiment of the present application, an optical flow method is used to calculate a motion vector of a second image frame relative to the first image frame in each of two adjacent image frames in the first image sequence, and the shooting time of the second image frame is later than the shooting time of the first image frame. The optical flow method may be a Lucas–Kanade optical flow algorithm, a KLT optical flow algorithm, a Farneback optical flow algorithm, an optical flow algorithm based on deep learning, and the like.

[0145] In one embodiment of the present application, after the motion vector of the second image frame relative to the first image frame is calculated, the motion vectors of all pixels of the target object and the target object's neighborhood are obtained according to a pixel window of a preset size, and the average of the motion vectors of all pixels of the target object and the target object's neighborhood is calculated to obtain the motion amplitude of the second image frame relative to the first image frame. For example, the preset size may be 3*3, 5*5 or other sizes.

[0146] In one embodiment of the present application, the average of the motion amplitudes of the second image frame relative to the first image frame in all adjacent image frames in the first image sequence is calculated to obtain the motion amplitude of the target object, wherein the motion amplitude of the target object is in pixel units and is used to describe the degree and direction of motion of the target object in the image.

[0147] S1053, determine whether the target object's motion amplitude is greater than or equal to a preset motion amplitude threshold. If the target object's motion amplitude is greater than or equal to the preset motion amplitude threshold, the process proceeds to S1054; if the target object's motion amplitude is less than the preset motion amplitude threshold, the process proceeds to S1055.

[0148] S1054: Acquire multiple image frames within a third preset time length from the first image sequence according to the reference image frame to generate a fourth image sequence.

[0149] See also Fig.14As shown, it is a schematic diagram of generating a fourth image sequence provided by an embodiment of the present application. In one embodiment of the present application, if the motion amplitude LV1 of the target object is greater than or equal to the preset motion amplitude threshold, a reference image frame, a plurality of image frames within the fifth preset time length before the reference image frame, and a plurality of image frames within the sixth preset time length after the reference image frame are obtained from the first image sequence to generate a fourth image sequence. For example, the preset motion amplitude threshold is 10 pixel units. Among them, the sum of the fifth preset time length and the sixth preset time length is the third preset time length. For example, the third preset time length is 0.75 seconds, the fifth preset time length is 0.5 seconds, and the sixth preset time length is 0.25 seconds.

[0150] S1055: Acquire multiple image frames within a fourth preset time length from the first image sequence according to the reference image frame to generate a fourth image sequence.

[0151] In one embodiment of the present application, if the motion amplitude LV2 of the target object is less than the preset motion amplitude threshold, a reference image frame, multiple image frames within the seventh preset time length before the reference image frame, and multiple image frames within the eighth preset time length after the reference image frame are obtained from the first image sequence to generate a fourth image sequence, wherein the fourth preset time length is greater than the third preset time length, and the sum of the seventh preset time length and the eighth preset time length is the fourth preset time length. For example, the fourth preset time length is 1.5 seconds, the seventh preset time length is 1 second, and the sixth preset time length is 0.5 seconds.

[0152] The target video generated by the embodiment of the present application is a time-lapse video or slow-motion video of exciting actions. Therefore, if the motion amplitude of the target object in the image frame is large, fewer image frames are obtained for interpolation processing. If the motion amplitude of the target object in the image frame is small, more image frames are obtained for interpolation processing.

[0153] In another embodiment of the present application, in S1053 and S1054, multiple image frames can be obtained from the second image sequence corresponding to the photo stream according to the same method to generate a fourth image sequence, thereby generating a video with a higher resolution.

[0154] The above-mentioned embodiments of the present application can estimate the motion speed of the target moving object through multiple image frames in the image stream, and obtain different numbers of image frames according to the motion speed of the target moving object, so that the generated video of the wonderful action is adapted to the motion speed of the target moving object.

[0155] See also Fig.15 As shown, it is a flowchart of generating a target video provided by an embodiment of the present application.

[0156] S1061, determine whether the target object's motion amplitude is greater than or equal to a preset motion amplitude threshold. If the target object's motion amplitude is greater than or equal to the preset motion amplitude threshold, the process proceeds to S1062; if the target object's motion amplitude is less than the preset motion amplitude threshold, the process proceeds to S1063.

[0157] S1062: Perform interpolation processing on a plurality of image frames in the fourth image sequence according to the first frame rate to generate a target video of a ninth preset duration.

[0158] In one embodiment of the present application, the first frame rate is 240fps (frame per second), and the ninth preset duration is 6 seconds. According to the first frame rate, a preset interpolation algorithm is used to interpolate multiple image frames in the fourth image sequence to generate a target video of the ninth preset duration.

[0159] In one embodiment of the present application, the preset interpolation algorithm may be a linear interpolation algorithm, a bilinear interpolation algorithm, or an interpolation algorithm based on optical flow. The linear interpolation algorithm is based on a linear relationship and generates an intermediate image frame by calculating the linear interpolation of pixels between two adjacent image frames. For example, the position ratio of each pixel point in two adjacent image frames is calculated, and the pixel values ​​of the two pixels are weighted averaged according to the position ratio to generate the pixel value of the corresponding pixel point in the intermediate frame. The bilinear interpolation algorithm performs a weighted average of the pixel values ​​of the four nearest pixels to generate an intermediate image frame. The interpolation algorithm based on optical flow uses an optical flow estimation method to predict the pixel displacement of the intermediate image frame, and then generates the pixel value of the corresponding pixel point in the intermediate image frame through interpolation calculation.

[0160] S1063, interpolating multiple image frames in the fourth image sequence according to the second frame rate to generate a target video of a ninth preset duration, wherein the first frame rate is greater than the second frame rate, for example, the second frame rate is 120 fps.

[0161] The above-mentioned embodiments of the present application can use different frame rates to interpolate multiple image frames according to the movement speed of the target moving object to generate a video of wonderful action, so that the generated video of wonderful action is adapted to the movement speed of the target moving object.

[0162] For example, see Fig.16 FIG. 4 shows a plurality of image frames in a fourth image sequence, wherein the plurality of image frames include image frame a, image frame b and image frame c. Fig.17As shown, multiple image frames after interpolation processing are generated through interpolation processing, and multiple image frames between image frame a and image frame b, and multiple image frames between image frame b and image frame c are respectively generated, and a target video is generated based on the generated multiple image frames, image frame a, image frame b, and image frame c. Compared with the actions in the multiple image frames in the fourth image sequence, the actions in the multiple image frames in the target video are richer and more coherent, which is suitable for showing the process of wonderful actions.

[0163] See also Fig.18 As shown, it is a schematic diagram of generating the fifth image sequence provided by one embodiment of the present application. In one embodiment of the present application, a reference image frame and a fourth preset number of image frames before the reference image frame are obtained from the second image sequence to generate the fifth image sequence. For example, the fourth preset number is 3. A preset photographing algorithm is executed on multiple image frames in the fifth image sequence to obtain a snapshot result. For example, the preset photographing algorithm includes but is not limited to a registration algorithm, a noise reduction algorithm, and a fusion algorithm. In another embodiment of the present application, a reference image frame, a fifth preset number of image frames before the reference image frame, and a sixth preset number of image frames after the reference image frame can also be obtained from the second image sequence to generate the fifth image sequence. For example, the fifth preset number is 3 and the sixth preset number is 2.

[0164] See also Fig.19 As shown, it is a flowchart of generating a target image provided by an embodiment of the present application.

[0165] S1081, register multiple image frames in the fifth image sequence.

[0166] In one embodiment of the present application, a scale-invariant feature transform (SIFT) algorithm is used to align multiple image frames in the fifth image sequence. First, scale space extremum detection is performed, and a scale space pyramid is constructed using a Gaussian filter for each image frame in the fifth image sequence. Key points are detected by Gaussian differences at different scales to extract stable scale-invariant features. Then, key point positioning is performed. In each scale space, key points are screened by comparing the gradient and curvature of pixels with their surrounding pixels, and local extreme points of the image are used to accurately locate and suppress edge responses. Next, direction allocation is performed, and a main direction is assigned to each key point so that the descriptor has rotation invariance, and a gradient direction histogram is calculated to select a dominant direction or multiple directions as the direction of the key point. Then, descriptor generation is performed, and a descriptor is constructed in a neighborhood near the key point using the scale and direction information of the key point. During the descriptor extraction process, a multi-dimensional (e.g., 128-dimensional) feature vector is generated by combining the local gradient and gradient direction of the image. Feature matching is then performed to match the feature descriptors of each two images. For example, the similarity between feature vectors is measured based on Euclidean distance or cosine similarity, and the best match is selected. Finally, registration is performed, and the transformation matrix between images is calculated based on the matched feature point pairs using a corresponding registration algorithm (such as the random sampling consensus algorithm RANSAC) to achieve image alignment and registration.

[0167] S1082: Perform noise reduction processing on multiple image frames in the fifth image sequence.

[0168] In one embodiment of the present application, a Laplacian-based pyramid algorithm is used to reduce noise for multiple image frames in the fifth image sequence. First, by performing multiple downsampling and Gaussian filtering on the original image, images of different scales are generated, and a corresponding Laplacian pyramid is constructed from each level in the Gaussian pyramid through upsampling and difference operations, wherein the Laplacian pyramid contains detailed information at each scale of the image. The pixel values ​​of each pyramid level are smoothed by using a mean filter or other filters, so that noise reduction is performed on each level of the Laplacian pyramid. The Laplacian pyramid that has been subjected to noise reduction is combined with the Gaussian pyramid, and the image frame after noise reduction is reconstructed by upsampling and weighted summing layer by layer.

[0169] S1083, fuse multiple image frames in the fifth image sequence to obtain a fused image.

[0170] In one embodiment of the present application, an image fusion algorithm is used to fuse multiple image frames after noise reduction in the fifth image sequence to obtain a fused image. For example, the image fusion algorithm can be a Laplace pyramid fusion algorithm, a wavelet transform fusion algorithm, a region segmentation and weighted average fusion algorithm, an image fusion algorithm based on a neural network, etc.

[0171] Through the above embodiments of the present application, the quality of the fused image can be improved by performing preprocessing of registration and noise reduction before image fusion.

[0172] In another embodiment of the present application, key features of multiple image frames in the fifth image sequence can be extracted, and image fusion can be performed based on the key features, so as to highlight the key features in the fused target image, so that the target image presents a blurred effect.

[0173] The present application also provides an electronic device 100. Fig. 20 As shown, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device 100.

[0174] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0175] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0176] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0177] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0178] The processor 110 may also be provided with a memory for storing instructions and data. In one embodiment of the present application, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use instructions or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0179] In one embodiment of the present application, the processor 110 may include one or more interfaces. The interface may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0180] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment of the present application, the processor 110 may include multiple groups of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 can be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0181] The I2S interface can be used for audio communication. In one embodiment of the present application, the processor 110 may include multiple I2S buses. The processor 110 may be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In one embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call via a Bluetooth headset.

[0182] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment of the present application, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In one embodiment of the present application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0183] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment of the present application, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In one embodiment of the present application, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0184] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In one embodiment of the present application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0185] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In one embodiment of the present application, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0186] The USB interface 130 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices 100, such as AR devices, etc.

[0187] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0188] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0189] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0190] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0191] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0192] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (Low Noise Amplifier, LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0193] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In one embodiment of the present application, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be arranged in the same device as the mobile communication module 150 or other functional modules.

[0194] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0195] In one embodiment of the present application, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include Global System For Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a Beidou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS).

[0196] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for shooting processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0197] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, Microled, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In one embodiment of the present application, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0198] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0199] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In one embodiment of the present application, ISP can be set in camera 193.

[0200] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In one embodiment of the present application, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0201] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0202] The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0203] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0204] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0205] Random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

[0206] Non-volatile memory may include disk storage devices and flash memory.

[0207] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle. It can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage specification. It can be divided into universal flash storage (UFS), embedded Multi Media Card (eMMC), etc. according to the storage specification.

[0208] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0209] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0210] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0211] The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor 110, the screen display detection method executed on the electronic device 100 in the above embodiment can be implemented to implement the screen display detection function of the electronic device 100.

[0212] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0213] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In one embodiment of the present application, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0214] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0215] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0216] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0217] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm Open Mobile Terminal Platform (OMTP) standard interface or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0218] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0219] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0220] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0221] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In one embodiment of the present application, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100. The embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the electronic device 100, the electronic device 100 executes the above-mentioned related method steps to implement the shooting processing method in the above-mentioned embodiment.

[0222] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the shooting processing method in the above-mentioned embodiment.

[0223] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the shooting processing method in the above-mentioned method embodiments.

[0224] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0225] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0230] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A shooting processing method, applied to electronic equipment, characterized in that: The method comprises: In response to a user opening a camera application, collecting an image stream through the camera of the electronic device; In response to a shooting operation of a user, generating a first image sequence according to the image stream; Acquire multiple image frames from the first image sequence to generate a second image sequence; selecting a reference image frame from the second image sequence based on a score for each image frame in the second image sequence; Acquire multiple image frames from the first image sequence according to the reference image frame to generate a third image sequence; Generate a target video according to a plurality of image frames in the third image sequence; Acquire multiple image frames from the first image sequence according to the reference image frame to generate a fourth image sequence; A target image is generated according to a plurality of image frames in the fourth image sequence.

2. The shooting processing method according to claim 1, characterized in that: The step of responding to a shooting operation of a user and generating a first image sequence according to the image stream includes: According to the moment when the user performs the shooting operation, image frames captured by the camera within a first preset time period before the moment and image frames captured by the camera within a second preset time period after the moment are acquired from the image stream to generate the first image sequence.

3. The shooting processing method according to claim 1, characterized in that: The acquiring a plurality of image frames from the first image sequence to generate a second image sequence comprises: One image frame is acquired from every first preset number of consecutive image frames in the first image sequence to generate the second image sequence.

4. The shooting processing method according to claim 1, characterized in that: The step of selecting a reference image frame from the second image sequence based on a score for each image frame in the second image sequence comprises: Inputting each image frame in the second image sequence into at least one scoring model to obtain at least one score for each image frame; The reference image frame is selected from the second image sequence based on the at least one score of each image frame.

5. The shooting processing method according to claim 4, characterized in that: The at least one scoring model includes an action scoring model, an image clarity scoring model, and an expression scoring model.

6. The shooting processing method according to claim 1, characterized in that: The method further comprises: identifying a target object in a plurality of image frames of the first image sequence; The motion amplitude of the target object is calculated according to a plurality of image frames in the first image sequence.

7. The shooting processing method according to claim 6, characterized in that: The identifying the target object in the plurality of image frames of the first image sequence comprises: The multiple image frames of the first image sequence are input into a target detection model, and the target objects in the multiple image frames of the first image sequence are output through the target detection model.

8. The shooting processing method according to claim 7, characterized in that: The calculating the motion amplitude of the target object according to the plurality of image frames in the first image sequence comprises: According to every two adjacent image frames in the first image sequence, an optical flow method is used to calculate the motion amplitude of the target object.

9. The shooting processing method according to claim 6, characterized in that: The acquiring a plurality of image frames from the first image sequence according to the reference image frame to generate a third image sequence comprises: If the motion amplitude of the target object is greater than or equal to a preset motion amplitude threshold, acquiring a plurality of image frames within a third preset time length from the first image sequence according to the reference image frame to generate the third image sequence; or If the motion amplitude of the target object is smaller than the preset motion amplitude threshold, multiple image frames within a fourth preset time length are acquired from the first image sequence according to the reference image frame to generate the third image sequence, and the third preset time length is smaller than the fourth preset time length.

10. The shooting processing method according to claim 9, characterized in that: The acquiring a plurality of image frames within a third preset time length from the first image sequence according to the reference image frame to generate the third image sequence includes: The third image sequence is generated by acquiring multiple image frames within a fifth preset time length before the reference image frame and acquiring multiple image frames within a sixth preset time length after the reference image frame from the first image sequence, wherein the sum of the fifth preset time length and the sixth preset time length is the third preset time length.

11. The shooting processing method according to claim 10, characterized in that: Generating a target video according to a plurality of image frames in the third image sequence comprises: If the motion amplitude of the target object is greater than or equal to the preset motion amplitude threshold, interpolating the plurality of image frames in the third image sequence according to the first frame rate to generate the target video of a ninth preset duration; or If the motion amplitude of the target object is less than the preset motion amplitude threshold, multiple image frames in the third image sequence are interpolated according to the second frame rate to generate the target video of a ninth preset duration, wherein the first frame rate is greater than the second frame rate.

12. The shooting processing method according to claim 9, characterized in that: The acquiring a plurality of image frames within a fourth preset time length from the first image sequence according to the reference image frame to generate the third image sequence includes: The third image sequence is generated by acquiring multiple image frames within a seventh preset time length before the reference image frame and acquiring multiple image frames within an eighth preset time length after the reference image frame from the first image sequence, wherein the sum of the seventh preset time length and the eighth preset time length is the fourth preset time length.

13. The shooting processing method according to claim 10, characterized in that: The method further comprises: storing the target video in a gallery application of the electronic device; The reference image frame is optimized, and the optimized reference image frame is used as the cover image of the target video.

14. The shooting processing method according to claim 1, characterized in that: The acquiring a plurality of image frames from the first image sequence according to the reference image frame to generate a fourth image sequence comprises: The reference image frame and a second preset number of image frames before the reference image frame are acquired from the first image sequence to generate the fourth image sequence.

15. The shooting processing method according to claim 14, characterized in that: Generating a target image according to a plurality of image frames in the fourth image sequence comprises: fusing a plurality of image frames in the fourth image sequence to obtain a fused image; The fused image is optimized to obtain the target image.

16. The shooting processing method according to any one of claims 1 to 15, characterized in that: The image stream includes a preview image stream and a photographing image stream, and a resolution of image frames in the photographing image stream is greater than a resolution of image frames in the preview image stream.

17. An electronic device, characterized in that: The electronic device comprises a memory and a processor: Wherein, the memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the electronic device executes the shooting processing method according to any one of claims 1 to 16.

18. A chip coupled to a memory in an electronic device, characterized in that: The chip is used to control the electronic device to execute the shooting processing method according to any one of claims 1 to 16.

19. A computer storage medium, characterized in that: The computer storage medium stores program instructions, and when the program instructions are executed on an electronic device, a processor of the electronic device executes the shooting processing method according to any one of claims 1 to 16.

Images