Video processing method, electronic device and readable medium

Abstract

The embodiment of the present application provides a video processing method, an electronic device and a computer-readable storage medium, the video processing method comprising: in response to a first operation, displaying a first interface and starting to shoot a first video; during the shooting process of the first video, obtaining the first video; identifying the theme, transition and wonderful moments of the first video, obtaining the label data of each transition of the first video and the label data of the first video; using the wonderful moments, capturing multiple wonderful images from the first video; in response to a second operation, saving the label data of each transition of the first video and the label data of the first video, and saving the wonderful images. It can be seen that the electronic device can obtain the first video shot, and the label data of the first video. The electronic device can also use the wonderful moments in the label data of the first video to capture multiple wonderful images from the first video, so that the user can obtain photos of wonderful moments while shooting the video.

Description

[0001] This application is a divisional application of the invention patent application with the application date of March 15, 2022, application number 202210255907.7, and invention name “Video processing method, electronic device and readable medium”. Technical Field

[0002] The present application relates to the technical field of electronic equipment, and in particular to a video processing method, electronic equipment, program product and computer-readable storage medium. Background Art

[0003] At present, the functions of taking photos and recording videos have become essential functions of electronic devices. Users' demand and experience for recording and taking photos are also increasing. In some application scenarios of shooting videos, users expect to capture memorable photos of wonderful moments while shooting videos.

[0004] Therefore, it is necessary to provide a method for obtaining memorable photos of wonderful moments while shooting a video. Summary of the invention

[0005] The present application provides a video processing method, an electronic device, a program product and a computer-readable storage medium, the purpose of which is to enable users to obtain photos of wonderful moments while shooting videos.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a video processing method for an electronic device, the video processing method comprising: in response to a first operation, displaying a first interface and starting to shoot a first video, the first interface being a shooting interface of a camera application, the first interface including a first control, the first control being used to stop shooting the first video; during the shooting of the first video, acquiring the first video; identifying the theme, transitions and wonderful moments of the first video, and obtaining label data of each transition of the first video and label data of the first video; the transitions are used to divide the first video into multiple video segments, the label data of each transition including: information of the video segment corresponding to the transition; the label data of the first video including the theme label, wonderful moments and group ID of the first video; using the wonderful moments, capturing multiple wonderful images from the first video; in response to a second operation, saving the label data of each transition of the first video and the label data of the first video, and saving wonderful images, the wonderful images including the group ID of the first video, the second operation being a trigger operation for the first control.

[0008] From the above content, it can be seen that when a user uses an electronic device to shoot a video, the electronic device can obtain the first video shot and the tag data of the first video. The electronic device can also use the wonderful moments in the tag data of the first video to capture multiple wonderful images from the first video, so that the user can obtain photos of wonderful moments while shooting a video.

[0009] In one possible implementation, the theme, transitions, and highlights of the first video are identified to obtain label data for each transition of the first video and label data for the first video, including: identifying the end of a transition in the first video and generating label data for the transition using information of a video clip corresponding to the transition; and generating label data for the first video using the theme, highlights, and group ID of the first video at the end of shooting the first video.

[0010] In one possible implementation, in response to the second operation, the tag data of each transition of the first video and the tag data of the first video are saved, including: in response to the second operation, the tag data of each transition and the tag data of the first video are written into the video header of the first video.

[0011] In one possible implementation, writing the tag data of each transition and the tag data of the first video into the video header of the first video includes: the media recorder receives the tag data of each transition and the tag data of the first video sent by the camera application; the media recorder writes the tag data of each transition and the tag data of the first video into the video header of the first video.

[0012] In one possible implementation, the theme, transitions, and highlights of the first video are identified, and label data of each transition of the first video and label data of the first video are obtained, including: camera Hal identifies the theme, transitions, and highlights of the first video, and obtains label data of each transition of the first video and label data of the first video.

[0013] In one possible implementation, the camera Hal identifies the theme, transitions, and highlights of the first video, and obtains the label data of each transition of the first video and the label data of the first video, and also includes: the camera Hal receives a first control command sent by the camera application, the first control command is obtained by the camera application in response to a first operation, and the first operation includes a user's touch operation on a control for controlling shooting.

[0014] In a possible implementation, using the wonderful moment to capture multiple wonderful images from the first video includes: capturing images of the wonderful moment from the first video; and selecting multiple images with high wonderfulness scores from the images of the wonderful moment as the wonderful images.

[0015] In a possible implementation, utilizing the wonderful moment to capture multiple wonderful images from the first video includes: the camera Hal utilizes the wonderful moment to capture multiple wonderful images from the first video.

[0016] In a possible implementation, the video processing method further includes: when the shooting of the first video is finished, the camera application sends the storage path of the first video to the media provider; and the media provider saves the storage path of the first video.

[0017] In one possible implementation, the video processing method also includes: the media information library obtains data saved by the media provider, the data including at least the storage path of the first video; the media information library obtains the first video using the storage path of the first video; the media information library obtains the group ID of the first video saved in the video header of the first video.

[0018] In one possible embodiment, the video processing method also includes: in response to a third operation, obtaining the first video, the third operation being a browsing operation of the first video; obtaining wonderful images in the first video; and displaying a browsing interface of the first video, the browsing interface of the first video including: thumbnails of the first video and the wonderful images in the first video.

[0019] In this possible implementation, when the user inputs a third operation to browse the first video, the electronic device can obtain wonderful images in the first video and display the first video and the wonderful images in the first video on the browsing interface of the first video, so that the user can view the video and the wonderful images in the video on the browsing interface of the video.

[0020] In one possible implementation, in response to a third operation, the first video is obtained, and wonderful images in the first video are obtained; a browsing interface for the first video is displayed, including: the gallery application obtains the first video in response to the third operation; the gallery application obtains wonderful images in the first video; and the gallery application displays a browsing interface for the first video.

[0021] In a possible implementation, the gallery application obtains the first video in response to the third operation, including: the gallery application obtains the storage path of the first video from the media information library in response to the third operation; and the gallery application obtains the first video using the storage path of the first video.

[0022] In a possible implementation, the video processing method further includes: in response to a generation operation of a selected video, acquiring a first video; and using the first video to generate a selected video, wherein the selected video includes images of wonderful moments of the first video.

[0023] In this possible implementation, the electronic device obtains the first video in response to the operation of generating a selected video, and uses the first video to generate a selected video, so that the user can obtain photos of wonderful moments while shooting the video, and further obtain a short video of the first video to facilitate user sharing.

[0024] In one possible implementation, in response to a selected video generation operation, a first video is acquired; and a selected video is generated using the first video, wherein the selected video includes images of the highlight moments of the first video, including: a video editing application acquires the first video in response to the selected video generation operation; and a video editing application generates a selected video using the first video, wherein the selected video includes images of the highlight moments of the first video.

[0025] In one possible implementation, the video editing application obtains the first video in response to the operation of generating the selected video, including: the video editing application receives the storage path of the first video sent by the gallery application in response to the operation of generating the selected video; the video editing application obtains the first video using the storage path of the first video.

[0026] In a possible implementation, the video processing method further includes: in response to a save operation of the selected video, saving the selected video.

[0027] In a second aspect, the present application provides an electronic device comprising: one or more processors, a memory, a camera and a display screen; the memory, the camera and the display screen are coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when one or more processors execute the computer instructions, the electronic device performs a video processing method such as any one of the first aspects.

[0028] In a third aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement a video processing method as described in any one of the first aspects.

[0029] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer executes a video processing method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Hardware structure diagram of the electronic device provided for this application;

[0031] Figure 2 A schematic diagram of an example of enabling "one record, multiple gains" provided in an embodiment of the present application;

[0032] Figure 3A schematic diagram of a graphical user interface of "one entry, multiple results" provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of another example of a "one-record, multiple-receive" graphical user interface provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of another example of a "one-record, multiple-receive" graphical user interface provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of another example of a "one-record, multiple-receive" graphical user interface provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of another example of a "one-record, multiple-receive" graphical user interface provided in an embodiment of the present application;

[0037] Figure 8 A schematic diagram of another example of a "one-record, multiple-receive" graphical user interface provided in an embodiment of the present application;

[0038] Fig. 9 A software framework diagram of an electronic device provided in an embodiment of the present application;

[0039] Fig.10 A signaling diagram of a video processing method provided in an embodiment of the present application;

[0040] Fig.11 Another signaling diagram of a video processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0042] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0043] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0044] Before introducing the embodiments of the present application, some terms or concepts involved in the embodiments of the present application are first explained. It should be understood that the present application does not specifically limit the naming of the following terms. The following terms may have other names. The renamed terms still meet the following related term explanations.

[0045] 1) One-shot multiple-record, can be understood as when a user uses the camera app to shoot a video, by pressing the "shoot" icon once, the user can get the original video, one or more wonderful photos, and one or more selected videos. It is understandable that the duration of the wonderful short video obtained by one-shot multiple-record is less than the duration of the entire video. For example, if the entire video recorded is 1 minute, 5 photos of wonderful moments and a wonderful short video of 15 seconds can be obtained. It is also understandable that one-shot multiple-record can also have other names, such as one-click multiple-record, one-click multiple shots, one-click film output, one-click blockbuster, AI one-click blockbuster, etc.

[0046] 2) Wonderful images refer to the pictures of some wonderful moments in the video recording process. For example, the wonderful image can be the best sports moment picture, the best expression moment picture or the best punch-in action picture. It is understandable that the present application does not limit the term wonderful image, and the wonderful image can also be called a beautiful moment image, a magical moment image, a wonderful moment image, a decisive moment image, a best shot (BS) image, or an AI image, etc. In different scenarios, wonderful images can be different types of moment pictures. For example, when shooting a football game video, the wonderful image can be an image of the moment when the athlete's foot touches the football when shooting or passing, an image of the moment when the football is kicked away by the athlete, and the wonderful image can also be an image of the moment when the football flies into the goal, and an image of the moment when the goalkeeper catches the football. When shooting a video of a person jumping from the ground, the wonderful image can be an image of the moment when the person is at the highest point in the air, or an image of the moment when the person is in the air and the most stretched action. When shooting scenery, the wonderful image can be an image of a building appearing in the scenery, or an image of the sunset or sunrise.

[0047] 3) Selected videos refer to videos containing wonderful images. It is understandable that the present application does not limit the term selected videos, and selected videos can also be called wonderful videos, wonderful short videos, wonderful small videos, or AI videos, etc.

[0048] 4) Tags (TAGs). Tags can be divided into theme tags, scene tags, wonderful image tags, and storyboard tags, etc. The theme tag is used to indicate the style or atmosphere of the video; the scene tag is used to indicate the scene of the video; the wonderful image tag is used to indicate the location of the wonderful image in the shot video, and the storyboard tag (also called transition tag) is used to indicate the location of the transition scene in the shot video. For example, a video includes one or more wonderful image tags, and the wonderful image tag can indicate that the image frame at the 10th second, 1 minute and 20 seconds of the video is a wonderful image. The video also includes one or more storyboard tags, and the storyboard tag can indicate that the video switches from the first scene to the second scene at the 15th second, and from the second scene to the third scene at the 3rd minute and 43 seconds.

[0049] At present, the functions of taking photos and recording videos have become essential functions of electronic devices. The user's demand and experience for recording and taking photos are also increasing. In some application scenarios of shooting videos, users expect to capture memorable photos of wonderful moments while shooting videos. Based on this, the present application sets a "one-record-many-gets" mode in the electronic device, that is, when the electronic device shoots a video in the recording mode, the wonderful images in the video stream are automatically extracted by analyzing the video stream. Moreover, when the video shooting is completed, the electronic device can also generate one or more selected videos. In addition, users can view the shot videos, wonderful images and selected videos in applications such as the gallery.

[0050] In order to support the "one record, multiple get" mode of electronic devices, the embodiment of the present application provides a video processing method. In addition, the video processing method provided in the embodiment of the present application can be applied to mobile phones, tablet computers, desktops, laptops, notebook computers, ultra-mobile personal computers (Ultra-mobile Personal Computer, UMPC), handheld computers, netbooks, personal digital assistants (Personal Digital Assistant, PDA), wearable electronic devices and smart watches, etc.

[0051] Take mobile phones as an example. Figure 1 This is an example of the composition of an electronic device provided in an embodiment of the present application. Figure 1 As shown, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display screen 140, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, and a button 190, etc.

[0052] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0054] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0055] The internal memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0056] In some embodiments, the internal memory 120 stores instructions for executing the video processing method. The processor 110 can control the electronic device to shoot a video in a "one-shot, multiple-shot" mode by executing the instructions stored in the internal memory 120, and obtain the shot video, one or more wonderful photos, and one or more selected videos.

[0057] The electronic device implements the display function through a GPU, a display screen 140, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 140 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0058] The display screen 140 is used to display images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oled, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 140, where N is a positive integer greater than 1.

[0059] In some embodiments, the electronic device shoots a video in a “record-all-inclusive” mode, obtaining the shot video, one or more wonderful photos, and one or more selected videos, which are displayed to the user by the display screen 140 .

[0060] The electronic device 100 can realize the shooting function through the ISP, the camera 130, the video codec, the GPU, the display screen 140 and the application processor.

[0061] The ISP is used to process the data fed back by the camera 130. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 130.

[0062] The camera 130 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 130, where N is a positive integer greater than 1.

[0063] In some embodiments, the camera 130 is used to shoot the video mentioned in the embodiments of the present application.

[0064] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0065] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as moving picture experts group (MPEG) 4, MPEG2, MPEG3, MPEG4, etc.

[0066] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0067] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0068] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0069] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0070] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0071] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0072] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device can listen to music or listen to a hands-free call through the speaker 170A.

[0073] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0074] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and directional recording function, etc.

[0075] The earphone interface 170D is used to connect a wired earphone and can be a USB interface, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0076] In the sensor module 180, the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 140, the electronic device detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0077] The touch sensor 180B is also called a "touch control device". The touch sensor 180B can be arranged on the display screen 140, and the touch sensor 180B and the display screen 140 form a touch screen, also called a "touch control screen". The touch sensor 180B is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 140. In other embodiments, the touch sensor 180B can also be arranged on the surface of the electronic device, which is different from the position of the display screen 140.

[0078] In some embodiments, the pressure sensor 180A and the touch sensor 180B may be used to detect a user's touch operation on a control, image, icon, video, etc. displayed on the display screen 140. The electronic device may respond to the touch operation detected by the pressure sensor 180A and the touch sensor 180B and execute a corresponding process. For the specific content of the process executed by the electronic device, please refer to the following embodiment.

[0079] The key 190 includes a power key, a volume key, etc. The key 190 can be a mechanical key or a touch key. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.

[0080] The technical solutions involved in the following embodiments can all be implemented in the electronic device 100 having the above hardware architecture.

[0081] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 Taking the electronic device with the structure shown as an example, the video processing method provided in the embodiment of the present application is specifically described.

[0082] The following embodiments of the present application will take the electronic device as a mobile phone, in which a camera application is installed, and the camera application is used to start the camera to shoot a video as an example to introduce in detail the video processing method provided by the present application.

[0083] In some embodiments of the present application, the user can manually turn on or off the "one record, multiple results" function provided by the embodiments of the present application. Figure 2 Describe the entrance to "record one, gain multiple benefits".

[0084] For example, the user can instruct the phone to open the camera application by touching a specific control on the phone screen, pressing a specific physical button or button combination, inputting voice, making air gestures, etc. Figure 2 (a) shows a way for a user to open the camera application. Figure 2As shown in (a), the user clicks the camera application icon displayed on the mobile phone display to input an instruction to turn on the camera. After the mobile phone receives the instruction from the user to turn on the camera, the mobile phone starts the camera and displays Figure 2 The shooting interface shown in (b) or (c).

[0085] Figure 2 The shooting interface shown in (b) is the shooting interface when the mobile phone is in camera mode. Figure 2 The shooting interface shown in (c) is the shooting interface when the mobile phone is in video recording mode. Figure 2 Taking (b) as an example, the shooting interface of the mobile phone includes: a control 201 for turning on or off the flash, a control 202 for settings, a switching list 203, a control 204 for displaying the last image taken, a control 205 for controlling shooting, and a control 206 for switching between the front and rear cameras.

[0086] The control 201 for turning on or off the flash is used to control whether the flash is turned on when the camera is shooting.

[0087] The setting control 202 can be used to set shooting parameters and shooting functions, such as setting the photo ratio, setting the gesture shooting, setting the smile capture, setting the video resolution, etc.

[0088] The switching list 203 includes multiple modes of the camera, and the user can switch between the multiple modes of the camera by sliding the switching list left and right. Figure 2 The switching list shown in (b) includes portrait, night scene, photo, video, and panorama. Others not shown in Figure 2 The mode in (b) is hidden display. The user can switch the list by sliding left and right to display the hidden display mode.

[0089] The control 204 for displaying the last captured image is used to display a thumbnail of the last captured image or a cover thumbnail of a video. The user can touch the control 204 for displaying the last captured image, and the display screen will display the last captured image or video. The last captured image or video refers to an image or video that was captured by the camera before the current capture and whose capture time is closest to the current capture time.

[0090] The control 205 for controlling shooting is a control provided to the user to start shooting. In the camera mode of the mobile phone, the user touches the control 205 for controlling shooting once, and the camera can shoot one frame of image. Of course, the camera can also shoot multiple frames of image, and only select one frame of image to output. In the video recording mode of the mobile phone, the user clicks the control 205 for controlling shooting, and the camera starts recording.

[0091] The control 206 for switching the front and rear cameras is used to switch the multiple cameras of the mobile phone. Generally, the mobile phone includes a camera on the same side as the display screen (referred to as the front camera) and a camera located on the housing of the mobile phone (referred to as the rear camera). The user can switch the front and rear cameras of the mobile phone by clicking the control 206 for switching the front and rear cameras.

[0092] User Figure 2 (b) or Figure 2 As shown in (c), by clicking the setting control 202, the mobile phone is controlled to display the setting interface. Figure 2 As shown in (d). Figure 2 In the setting interface shown in (d), option 207 for turning on "record multiple times" is displayed, which is used to turn on the function of recording multiple times. That is, when the user turns on this function, when the mobile phone is in the video recording mode to shoot a video, the mobile phone will automatically use the video processing method provided in the embodiment of the present application to automatically generate wonderful images and short videos while shooting the video. Of course, the user can also manually turn off the function of recording multiple times in the video recording mode through option 207.

[0093] Typically, Figure 2 The option 207 of "record one, get multiple results" shown in (d) is enabled by default. That is, when the mobile phone is turned on for the first time or the system is updated to have the "record one, get multiple results" function, Figure 2 The option 207 of "one record, multiple results" in the setting interface shown in (d) is in the turned-on state, and the "one record, multiple results" function of the mobile phone is activated.

[0094] It is understandable that the settings interface may also include controls for other function settings. For example, Figure 2 The photo setting controls and video setting controls shown in (d) include: the photo setting controls include: photo ratio setting controls, voice-controlled photo setting controls, gesture photo setting controls, smile capture setting controls, etc.; the video setting controls include: video resolution setting controls, video frame rate setting controls, etc.

[0095] The above describes a method for controlling a mobile phone to enter the "one record, multiple get" mode, but the present application is not limited to entering the "one record, multiple get" mode in the video recording mode. In some embodiments of the present application, the user may enable the "one record, multiple get" function in other ways.

[0096] After turning on the "record multiple times" function, users can control the phone to start shooting videos.

[0097] For example, see Figure 2 In (a), the user can click the camera application to start the camera to start collecting images and display a preview interface to the user. Figure 3As shown in (a), clicking the control 301 for controlling shooting controls the mobile phone to start recording the image collected by the camera. The mobile phone starting to record the image collected by the camera can be understood as: recording the image collected by the camera and saving it to the internal memory. Figure 3 The interface shown in (b) shows a user using a mobile phone to shoot a picture of a football game. Figure 3 The interface shown in (b) includes a stop control 302, a pause control 303, and a camera key 304. During the video shooting process, the user can pause the shooting by clicking the pause control 303, end the shooting by clicking the stop control 302, and manually capture a photo by clicking the camera key 304.

[0098] like Figure 3 In the interface shown in (c), the user can click the stop control 302 at 56 seconds to end the shooting process, and a video of 56 seconds in length can be obtained. After the shooting is completed, the display screen of the mobile phone can enter the shooting interface. In addition, when the "one record, multiple results" function of the mobile phone is used for the first time, a prompt for generating one record, multiple results can be guided in the shooting interface. For example, Figure 3 The shooting interface shown in (d) shows a dialog box for generating a prompt for "One shot, multiple results". The dialog box shows a text prompt "A wonderful photo of "One shot, multiple results" has been generated and you can create a blockbuster with one click". The user can click Figure 3 Any area of ​​the interface shown in (d) controls the dialog box of the multiple-record generation prompt to disappear. Alternatively, the mobile phone can also be configured to display the dialog box of the multiple-record generation prompt for a certain length of time, such as 5 seconds, and then disappear automatically.

[0099] Of course, after the "record one, get multiple results" function of the mobile phone is activated once, when the user shoots a video again and clicks the stop control to end the shooting, the shooting interface displayed on the phone's display screen will not include the prompt to guide the generation of "record one, get multiple results".

[0100] It should be noted that if the length of the video shot by the mobile phone meets certain length requirements, such as 15 seconds, the mobile phone will capture one or more wonderful images from the video shot by the mobile phone in the "one record, multiple get" mode, and generate a selected video. Figure 3 The shooting interface shown in (d) can display a dialog box for generating a prompt of "one shot, multiple results". The dialog box displays a text prompt "The "one shot, multiple results" wonderful photos have been generated and you can create a blockbuster with one click."

[0101] However, the application scenarios in which users shoot videos may also include the following two application scenarios: The first application scenario: the video shooting time is short and does not meet the video shooting time requirement of the one-record-multiple-records mode. The second application scenario: the video shooting time meets the video shooting time requirement of the one-record-multiple-records mode, but the mobile phone does not recognize a wonderful image from the shot video. In addition, in the second application scenario, the user is usually required to shoot for a time greater than another time requirement, which is greater than the video shooting time of the one-record-multiple-records mode, such as 30 seconds. Moreover, although the mobile phone does not recognize a wonderful image from the shot video, it can recognize images of better quality, which can be used to generate the selected videos proposed below.

[0102] In the first application scenario, when the user first shoots a video in the one-shot-multiple-records mode, Figure 4 As shown in (a), when the shooting reaches 10 seconds, the stop control 302 is clicked, and the shooting interface displayed on the mobile phone is as follows Figure 4 As shown in (b), the text of the dialog box for the "One-click, Multiple-Get" generation prompt is: The "One-click, Multiple-Get" wonderful photos have been generated.

[0103] In the second application scenario, when the user first shoots a video in the one-shot-multiple-records mode, Figure 4 As shown in (c), when the shooting reaches 56 seconds, the stop control 302 is clicked, and the shooting interface displayed on the mobile phone is as follows Figure 4 As shown in (d), the text of the dialog box for the "One Record, Many Gains" generation prompt is: "One Record, Many Gains" video can create blockbuster movies with just one click.

[0104] In addition, mobile phones Figure 4 After the shooting interface of the mobile phone is displayed in (b), the mobile phone can also confirm for the first time that the mobile phone has shot a video that meets the second application scenario when the user clicks Figure 4 After the stop control 302 shown in (c), display once Figure 4 Alternatively, when the mobile phone first determines that a video that meets the video shooting time requirement of the one-shot-multiple-records mode and can be identified as a wonderful image is shot, the mobile phone, when the user clicks Figure 3 After the stop control 302 shown in (c), display once Figure 3 The shooting interface is shown in (d).

[0105] In addition, the mobile phone Figure 4 After the shooting interface of the mobile phone is displayed in (d), when the mobile phone first determines the video that meets the first application scenario, the mobile phone can also Figure 4 After the stop control 302 shown in (a), display once Figure 4Alternatively, when the mobile phone determines for the first time that a video that meets the video shooting time requirement of the one-shot-multiple-records mode and can be identified as a wonderful image is shot, the mobile phone, when the user clicks Figure 3 After the stop control 302 shown in (c), display once Figure 3 The shooting interface is shown in (d).

[0106] It should be noted that when a user shoots a video in the "one-record-many" mode of a mobile phone, the mobile phone can use a recognition model to automatically identify wonderful images in the shot video. In some embodiments, the mobile phone is provided with a recognition model for wonderful images. During the process of shooting a video, the video is input into the recognition model for wonderful images. The recognition model can score the wonderfulness of the image in the input video to obtain a wonderfulness score value of the image in the video. The mobile phone can use the wonderfulness score value of the image to determine the wonderful image in the video. Generally, the larger the wonderfulness score value of the image, the higher the probability that the image is a wonderful image.

[0107] In some embodiments, the mobile phone may be configured to obtain a fixed number of wonderful images, such as 5 wonderful images. Based on this, the mobile phone selects 5 images with higher wonderfulness scores as wonderful images.

[0108] In some other embodiments, the mobile phone may not be configured with a limit on the number of wonderful images, and the mobile phone may select images with wonderfulness scores higher than a certain value as wonderful images.

[0109] After the mobile phone finishes shooting a video, the mobile phone obtains the shot video and one or more wonderful images in the video. The mobile phone can also save the shot video and wonderful images to the gallery. In an example, Figure 5 (a) shows the album display interface of the gallery, which displays all photos and videos saved in the mobile phone in the form of folders. Figure 5 The album display interface shown in (a) includes: a camera folder 401, an all photos folder 402, a video folder 403, and a one-shot folder 404. Of course, the album display interface may also include other folders, and this application does not limit the folders displayed in the camera display interface.

[0110] Typically, the camera folder 401 includes all photos and videos taken by the phone's camera, the all photos folder 402 includes all photos and videos saved by the phone, the video folder 403 includes all videos saved by the phone, and the one-shot folder 404 includes all wonderful images taken in one-shot mode and saved by the phone.

[0111] Figure 5(b) shows the display interface of the one-shot folder, which displays thumbnails of five wonderful images in the aforementioned 56-second video shot by the mobile phone in one-shot, and the five wonderful images are automatically captured by the mobile phone from the shot 56-second video.

[0112] After the mobile phone saves the captured video and the wonderful images in the video in the gallery, the user can view the video and the wonderful images in the video through the gallery application. Figure 6 In the shooting interface shown in (a), the user clicks the control 501 for displaying the image taken last time, or the user Figure 6 In the photo display interface of the gallery shown in (b), click on the cover thumbnail of video 502. In response to the user's click operation, the mobile phone displays a browsing interface of video 502 on the display screen of the mobile phone.

[0113] Figure 6 The video 502 shown in (b) has a video-specific corner mark for the "one record, multiple results" function displayed in the lower left corner of the cover thumbnail. The video-specific corner mark is used to indicate to the user that the video 502 corresponding to the cover thumbnail was shot by the mobile phone using the "one record, multiple results" function.

[0114] Figure 6 The cover thumbnail of the video 502 shown in (b) is different in size from the thumbnails of image A and image B. Image A and image B may refer to images taken by the mobile phone without the "record multiple times" function turned on. Figure 6 The cover thumbnail of video 502 shown in (b) is larger than the thumbnails of image A and image B. Of course, Figure 6 The cover thumbnail of the video 502 shown in (b) may also be the same as the thumbnails of other photos and videos in the gallery (referring to videos for which the one-record-to-many function is not enabled), and this embodiment of the present application is not limited to this.

[0115] The display screen of the mobile phone displays the browsing interface of video 502 for the first time, and a mask is displayed on the browsing interface. The display screen of the mobile phone displays the browsing interface of video 502 for the second time, and no mask is displayed on the browsing interface. For example, the browsing interface of video 502 with a mask is displayed as follows: Figure 6 As shown in (c), the browsing interface of the video 502 without the mask guide is as shown in FIG. Figure 6As shown in (d). It should also be pointed out that video 502 can be understood as the video shot by the mobile phone in the one-shot mode for the first time. Therefore, the first display of the browsing interface of video 502 on the mobile phone's display screen can be understood as the first display of the video shot by the mobile phone in the one-shot mode for the first time. In this case, the browsing interface of the video will be displayed with a mask guide. In addition, the first display of the video shot by the mobile phone in the one-shot mode for the first time by the mobile phone, and the display of the mask guide on the browsing interface of the video can serve to remind the user of the one-shot mode.

[0116] Figure 6 The browsing interface of the video 502 shown in (c) includes: a thumbnail area 504 of wonderful images of the video 502, a control 505, and a play control 506. Among them, the thumbnail area 504 of the wonderful images of the video 502 is exposed and not covered by the mask, and other areas of the display screen are covered by the mask.

[0117] The thumbnail area 504 of the wonderful images of video 502 includes: a cover thumbnail of video 502, and thumbnails of multiple wonderful images of video 502. The cover thumbnail of video 502 is usually in the first place, and the thumbnails of multiple wonderful images can be arranged according to the shooting time of the wonderful images, and are located after the cover thumbnail of video 502. The wonderful images can be, as mentioned above, automatically identified when the mobile phone shoots the video, and the wonderful moments contained in the video are extracted from the video. In addition, the browsing interface of video 502 also includes a reminder dialog box, which displays the text "Record one, get multiple" to intelligently capture multiple wonderful moments for you. The reminder dialog box is usually like this: Figure 6 As shown in (c), it is located above the thumbnail area 504 of the wonderful images of the video 502, and is used to prompt the user of the content displayed in the thumbnail area 504 of the wonderful images of the video 502, so as to guide the user to view the wonderful images of the video 502. Of course, Figure 6 The text and setting location displayed in the reminder dialog box shown in (c) are exemplary and do not constitute a limitation on the reminder dialog box. Users can click Figure 6 In any area of ​​the browsing interface of the video 502 shown in (c), the reminder dialog box is controlled to disappear. Alternatively, the mobile phone can also be configured to display the reminder dialog box for a certain length of time, such as 5 seconds, and then disappear automatically.

[0118] Control 505 is used to generate a selected video based on the wonderful images in video 502 .

[0119] The playback control 506 is used to control the playback of the video 502. For example, Figure 6As shown in (d), the play control 506 includes: a start or stop control 507, a slidable progress bar 508, and a speaker control 509. The start or stop control 507 is used to control the play of the video 502 or stop the play of the video 502; the speaker control 509 is used to select whether to play the video 502 in silence. The slidable progress bar 508 is used to display the play progress of the video 502, and the user can also adjust the play progress of the video 502 by dragging the circular control on the progress bar left and right.

[0120] The browsing interface of video 502 also includes options such as share, favorite, edit, delete, and more. If the user clicks share, the video 502 can be shared; if the user clicks favorite, the video 502 can be favorited in a folder; if the user clicks edit, the video 502 can be edited; if the user clicks delete, the video 502 can be deleted; if the user clicks more, other video operation functions (such as move, copy, add notes, hide, rename, etc.) can be entered.

[0121] The browsing interface of video 502 also includes the shooting information of video 502, which is usually as follows: Figure 6 As shown in (c) or (d), it is located above the video 502. The shooting information of the video 502 includes: the shooting date, shooting time and shooting address of the video 502. In addition, the browsing interface of the video 502 may also include a circular control, and the letter "i" is filled in the circular control. When the user clicks the circular control, the mobile phone can respond to the user's click operation and display the attribute information of the video 502 on the browsing interface of the video 502. For example, the attribute information may include the storage path, resolution, and configuration information of the camera when shooting the video 502.

[0122] Figure 6 The mask shown in (c) is a type of mask layer. A mask layer generally refers to a layer mask, which is a glass layer covering the layer of the interface displayed on the display screen. In addition, the glass layer is divided into transparent, semi-transparent, and completely opaque. Semi-transparent and completely opaque mask layers can block the light of the display screen, making the interface displayed on the display screen vaguely visible or completely invisible to the user. Figure 6 The mask shown in (c) can be understood as a translucent glass sheet layer.

[0123] Figure 6 The mask guide shown in (c) is an exemplary display and does not constitute a limitation of the mask guide for the first display of the browsing interface of the video shot by multiple recordings. In some embodiments, the mask guide can also be set as a guide with a mask and other special effects, such as a mask plus bubbles.

[0124] In addition, users can Figure 6(c) Input operation in any area of ​​the browsing interface of the video 502 to control the mask to disappear. Of course, the mobile phone can also be set to display the mask for a certain length of time, such as 3 seconds to disappear automatically. Figure 6 After the mask disappears in the browsing interface of the video 502 shown in (c) and the reminder dialog box disappears, the browsing interface of the video 502 is as follows: Figure 6 As shown in (d).

[0125] It should be noted that Figure 6 Before the mask on the browsing interface of the video 502 shown in (c) disappears, the video 502 is in a static state and will not play. After the mask disappears, the video 502 can be played automatically. Usually, the video 502 can also be played in silence. Of course, the user can click Figure 6 The speaker control 509 shown in (d) controls the mobile phone to play the video 502 with sound.

[0126] Users can also Figure 6 In some embodiments, the user clicks on a thumbnail of a wonderful image in the thumbnail area 504, and the mobile phone responds to the user's click operation and displays the wonderful image clicked by the user on the display screen to replace the thumbnail area 504 of the video 502 shown in (d). Figure 6 In another embodiment, when the user slides left or right in the thumbnail area 504, the mobile phone can also respond to the user's sliding operation and display the wonderful images in the thumbnail area 504 on the display screen in the direction of the user's sliding. The thumbnails of the wonderful images displayed in the thumbnail area 504 are not saved in the gallery, but in the album with multiple copies. That is, in Figure 6 In the interface shown in (b), there are no thumbnails corresponding to the wonderful images, but when the user clicks on the thumbnail of video 502 to enter the browsing interface of video 502, thumbnails of the wonderful images associated with video 502 can be displayed below the browsing interface of video 502.

[0127] Users can also Figure 6 In the video 502 shown in (d), a left-right sliding operation is input, and the mobile phone responds to the user's sliding operation and displays other images or videos stored in the mobile phone's gallery on the display screen. Figure 6 When the video 502 shown in (d) is swiped to the right, the mobile phone displays the next video or image of the video 502 saved in the gallery on the display screen. Figure 6When the video 502 shown in (d) is swiped to the left, the mobile phone displays the previous video or image of the video 502 saved in the gallery on the display screen. The previous video or image refers to a video or image shot earlier than the video 502 and most recently shot the video 502, and the next video or image refers to a video or image shot later than the video 502 and most recently shot the video 502.

[0128] It should be noted that if the video 502 shot by the user is a video of the first application scenario mentioned above, the browsing interface of the video 502 is the same as Figure 6 (c) and Figure 6 The difference is that the browsing interface of video 502 does not include control 505. If the video 502 shot by the user is a video of the second application scenario mentioned above, the browsing interface of video 502 is different from Figure 6 (c) and Figure 6 There is also some difference in (d), the difference is that the browsing interface of video 502 does not include thumbnail area 504.

[0129] It should also be noted that when a user shoots a video in the "one record, multiple get" mode of a mobile phone, the mobile phone can obtain the shot video, one or more wonderful images in the video, and tag data, which includes multiple tags (TAGs) of the video. In addition, the tag data can be added to the video, usually in the video header.

[0130] In a possible implementation, multiple tags (TAGs) of a video may be determined based on the hierarchical information of the video. The hierarchical information of the video may include: first-level information LV0, second-level information LV1, third-level information LV2, and fourth-level information LV3. Among them:

[0131] The first level information LV0 is used to characterize the theme category of the video and to give the style or atmosphere TAG of the entire video.

[0132] The second level information LV1 is used to characterize the scene of the video and to provide the scene TAG of the video.

[0133] The third level information LV2 is used to characterize changes in the scene of the video, which can also be understood as changes in the transition storyboard. The information of the third level information LV2 can give the video transition position (for example, the frame number where the transition occurs), as well as the transition type (character protagonist switching, fast camera movement, scene category change, and image content changes caused by other situations) to prevent too many similar scenes from being recommended. The information of LV2 is used to characterize video scene changes (or can also be referred to as transitions), including but not limited to one or more of the following changes: changes in the main body of the character (or protagonist), major changes in the image content composition, changes in the semantic level scene, and changes in image brightness or color. The mobile phone can use the third level information LV2 to add a storyboard TAG to the video when a scene change occurs in the video.

[0134] The fourth level information LV3 is used to represent the wonderful moment, that is, the shooting moment of the wonderful image, and is used to give the wonderful image TAG of the video.

[0135] The first level information LV0, the second level information LV1, the third level information LV2 and the fourth level information LV3 provide decision information in order of granularity from coarse to fine, so as to identify wonderful images in the video and generate a selected video.

[0136] Table 1 below gives examples of the definitions of LV0 and LV1.

[0137] Table 1

[0138]

[0139]

[0140] The mobile phone can use the captured video and tag data to generate a selected video of the captured video. Of course, when the mobile phone can identify wonderful images from the captured video, the selected video includes the wonderful images of the captured video with some special effects and music. In addition, if the mobile phone fails to identify wonderful images from the captured video, but can identify images of good quality, the selected video generated by the mobile phone includes images of good quality. Of course, the selected video also has some special effects and music. It should also be noted that the tag data will also include the TAG of the good quality image, which is used to indicate the shooting time of the good quality image, that is, the position in the captured video.

[0141] The good quality image mentioned in this application means: the image is relatively clear, for example, the resolution is high; or the image is relatively complete.

[0142] The special effects mentioned in this application refer to: special effects that can be supported by materials and added to video frames to present special effects, such as animation effects such as snowflakes and fireworks, as well as filters, stickers, frames, etc. In some embodiments, special effects can also be called styles or style themes.

[0143] The following content of this application is introduced using the example of a mobile phone using wonderful images to generate selected videos.

[0144] For example, a user Figure 7 As shown in (a), a control 601 of the video browsing interface is clicked. The mobile phone responds to the user's click operation and generates a selected video of a certain length.

[0145] Usually, it takes a certain amount of time to generate selected videos. Figure 7 As shown in (a), after clicking the control 601 on the video browsing interface, the display screen of the mobile phone will display the following Figure 7 After the selected video 602 is generated, the display screen of the mobile phone displays the display interface of the selected video 602. Of course, if the performance of the mobile phone is strong, the user can Figure 7 As shown in (a), after clicking the control 601 on the video browsing interface, the display screen of the mobile phone may not display Figure 7 The buffering interface of (b) is omitted, and the display interface of the selected video 602 is directly displayed.

[0146] For example, in the display interface of the selected video 602, the selected video 602 is being played. And the display interface of the selected video 602 is as follows: Figure 7 As shown in (c), it includes: style control 603, save control 604, share control 605, music control 606, edit control 607, etc.

[0147] The user clicks on the style control 603, and the mobile phone responds to the user's click operation, and the display screen displays a variety of video styles saved in the mobile phone. The user can select different video styles for the selected video 602. In some embodiments, the video style can be a filter, that is, the selected video 602 is color-adjusted by applying a filter. A filter is a type of video special effect, which is used to achieve various special effects for the selected video 602. In other embodiments, the video style can also be video effects such as fast forward and slow forward. In other embodiments, the video style can also refer to various themes, and different themes include corresponding filters, music and other content.

[0148] The user clicks on the share control 605 , and the mobile phone can share the selected video 602 in response to the user's click operation.

[0149] The user clicks the music control 606, and the mobile phone responds to the user's click operation, and the display screen displays an interface for adding different soundtracks to the wonderful video 602. The interface displays multiple soundtrack controls, and the user can click any soundtrack control to select a soundtrack for the wonderful short video, such as soothing, romantic, warm, cozy, tranquil, etc., to add a soundtrack to the selected video 602.

[0150] The user clicks the editing control 607, and the mobile phone responds to the user's click operation, and the display screen displays the editing interface of the wonderful video 602. The user can input editing operations such as editing, segmentation, volume adjustment, and frame adjustment on the wonderful video 602 in the editing interface.

[0151] The save control 604 is used to save the selected video 602 .

[0152] The following combination Figure 8 , the saving process of selected videos is introduced.

[0153] For example, the user clicks Figure 8 The save control 604 of the display interface of the selected video 602 shown in (a) is shown in FIG. The mobile phone saves the selected video 602 in response to the user's click operation. For example, Figure 8 As shown in (c), the mobile phone saves the selected video 602 in the gallery. In some embodiments, the selected video 602 can be saved along with the original video 502 of the selected video 602, that is, the selected video 602 and the original video 502 are stored in the same storage area. In other embodiments, the selected video 602 may not be saved along with the original video 502 of the selected video 602, that is, the selected video 602 and the original video 502 are stored in different storage areas. It should be noted that the original video 502 refers to the video shot by the user, and the selected video 602 comes from the original video 502.

[0154] User clicks Figure 8 The save control 604 of the display interface of the selected video 602 shown in (a) of FIG. In response to the user's click operation, the mobile phone saves the selected video 602 and the display screen of the mobile phone can also display the browsing interface of the selected video 602. For example, Figure 8 (b) shows a browsing interface of the selected video 602. In some embodiments, in the browsing interface of the selected video 602, the selected video 602 can be automatically played.

[0155] The browsing interface of the featured video 602, Figure 6 The browsing interface of the video 502 shown in (d) is basically the same, except that the browsing interface of the selected video 602 plays the selected video 602. The controls and their functions included in the browsing interface of the selected video 602 can be found in the aforementioned content. Figure 6The content of the browsing interface of the video 502 shown in (d) will not be described in detail here.

[0156] It should be noted that when the user clicks Figure 8 Before the save control 604 shown in (a) is clicked, the mobile phone can generate the selected video 602 but not save it in the internal memory. Figure 8 After clicking the save control 604 shown in (a), the mobile phone will also save the generated selected video 602 in the internal memory.

[0157] Fig. 9 It is a schematic diagram of the software structure of the electronic device proposed in the embodiment of the present application.

[0158] The layered architecture divides the operating system of the electronic device into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system of the electronic device is the Android system. The Android system can be divided into five layers, from top to bottom, namely, the application (APP) layer, the application framework layer (abbreviated as FWK), the system library, the hardware abstraction layer (HAL) and the kernel layer.

[0159] The application layer can include a series of application packages. Fig. 9 As shown, the application package may include applications such as camera, gallery, media library, video editing, map, call, calendar, etc.

[0160] In some embodiments, the camera is used to capture images or videos. Of course, the camera can respond to user operations to capture videos in a one-record-many mode.

[0161] The media information library is used to provide information such as the storage path and name of the videos and images in the gallery application. The media information library can also store some extended information of the videos and images. In some embodiments, the media information library stores the group ID and video type of the videos shot by the electronic device in a one-record-many-receive mode.

[0162] In addition, the media information library can also synchronize the information stored in the media provider (Media Provider). The information stored in the media information library can also be provided to the gallery application, and the gallery application reads the information in the media information library and displays the information for the user to view and operate. In some embodiments, the Media Library can also be set in the application framework layer.

[0163] The video editing application is used to provide functions related to video editing, such as video effects, filters, background music, clip editing, and other video editing functions, as well as video saving, video playback, and other video processing functions. In some embodiments, the video editing application can edit selected videos.

[0164] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Fig. 9 As shown, the application framework layer may include a window manager, a content provider, a phone manager, a resource manager, a camera post-processing path (Post Camera), a camera framework (CameraFwk), a media recorder (Media Recorder), a media provider (Media Provider), and the like.

[0165] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0166] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0167] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0168] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0169] The camera framework (CameraFwk) can also be called the interface corresponding to the camera application, which is used to provide data transmission function between the camera application and the lower-level modules. Normally, the camera application can send data to the lower-level module through the camera framework to control the lower-level module to collect video or images, and the video or image collected by the lower-level module can also be uploaded to the camera application through the camera framework. In some embodiments, the camera application can also send instructions to the camera Hal through the camera framework to control the camera Hal to analyze the video or stop analyzing the video. Of course, the data obtained by the camera Hal analyzing the video can also be uploaded to the camera application through the camera framework.

[0170] The camera post-processing path (Post Camera) is used to transmit the image data obtained by the lower layer module to the corresponding module of the application layer. In some embodiments, the image data of the wonderful image in the video can be transmitted to the camera application of the application layer through the camera post-processing path.

[0171] Media Recorder is used to record videos.

[0172] The Media Provider is used to provide the media information library with basic information about the video recorded by the media recorder.

[0173] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. In some embodiments of the present application, the application cold start will run in the Android runtime, and the Android runtime will obtain the optimized file status parameters of the application. Then, the Android runtime can determine whether the optimized file is outdated due to system upgrades based on the optimized file status parameters, and return the judgment result to the application management module.

[0174] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0175] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0176] The system library may include multiple functional modules, such as a surface manager, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0177] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0178] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms.

[0179] In some embodiments, such as Fig. 9 As shown, the hardware abstraction layer includes a camera Hal. The camera Hal may include functional modules such as an encoding module, a capture queue, and an algorithm recognition module.

[0180] The algorithm recognition module is an algorithm module that can load AI algorithms, and is used to provide the recognition function of the image content of the video, as well as the theme recognition, scene recognition, wonderful moment recognition, user operation and camera movement recognition of the video. In some embodiments, the algorithm recognition module may not belong to the camera Hal, but a functional module of the hardware abstraction layer (HAL).

[0181] The capture queue is used to save the wonderful images recognized by the algorithm recognition module from the video.

[0182] The encoding module is used to edit the images in the capture queue. In some embodiments, the wonderful images stored in the capture queue may be RAW images, and the encoding module edits the RAW images to obtain images in JPEG format or other formats.

[0183] The kernel layer is a layer between hardware and software. The kernel layer includes at least display driver, camera driver, sensor driver, audio driver, etc. In some embodiments, the display driver is used to control the display screen to display images; the camera driver is used to control the operation of the camera. The sensor driver is used to control the operation of multiple sensors, such as controlling the operation of pressure sensors and touch sensors.

[0184] The hardware layer may include the aforementioned foldable electronic device hardware components. Fig. 9 The display and camera are shown.

[0185] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS and Windows.

[0186] The following combination Fig. 9 , the process of the video processing method provided in the embodiment of the present application is introduced.

[0187] The video processing method provided in the embodiment of the present application can be divided into the following two steps: the first step is to shoot a video in a one-record-multiple-receive mode to obtain the shot video and the wonderful images in the video; the second step is to review the shot video and the wonderful images in the video to generate and save the selected video. Fig.10 and Fig.11 The specific implementation process of the above two links is introduced respectively.

[0188] Fig.10 Shown Fig. 9The signaling diagram of multiple working modules in the software framework of the electronic device shown cooperate to execute the first stage process. Fig.10 The method proposed in the first link for shooting a video in a one-record-multiple-receive mode to obtain the shot video and the wonderful images in the video includes the following steps:

[0189] S101: A camera application receives a first operation, where the first operation is used to start recording.

[0190] User Figure 3 As shown in (a), a control 301 for controlling shooting is clicked to input a first operation. The camera application receives the first operation and analyzes the first operation to determine that the operation is used to start recording.

[0191] It should be noted that if the user Figure 2 As shown in (a), click the camera application. The phone responds to the click operation and displays the shooting interface. The phone's camera is synchronously controlled to shoot a video, and the shooting interface can display the shot video to form a video preview interface. It should be pointed out that the phone only displays the video shot by the camera in the video preview interface, but the video shot by the camera is not saved. Therefore, the user's operation of clicking the camera application can be understood as starting the preview recording.

[0192] After the camera application receives the first operation, the camera application can control the start of recording the video taken by the camera, and of course can also be used to control the mobile phone to save the video.

[0193] S102: In response to the first operation, the camera application generates a UUID as a group ID.

[0194] Since the video is shot in the one-record-multiple-receive mode, the shot video, the wonderful images in the video, and the selected video can be obtained. Therefore, it is necessary to associate the shot video, the wonderful images in the video, and the selected video so that when the video is displayed, the wonderful images in the video can be displayed synchronously.

[0195] In some embodiments, the electronic device associates the video and the wonderful images in the video with the same group identity document (ID), and the electronic device can determine the video and the wonderful images belonging to the video through the group ID.

[0196] In some other embodiments, the electronic device associates the video, the wonderful images in the video, and the selected videos with the same group identity document (ID). The electronic device can determine the video, the wonderful images belonging to the video, and the selected videos corresponding to the video through the group ID.

[0197] The Universally Unique Identifier (UUID) can be used as a group ID. Based on this, when the camera application parses the first operation and determines that the user wants to shoot a video, the camera application generates a UUID in response to the first operation. It should be noted that the camera application needs to generate different UUIDs for different videos shot by the user.

[0198] It should be noted that before the camera application generates the UUID in response to the first operation, the camera application may first determine that the one-record-many function of the electronic device is in an activated state. In some embodiments, the electronic device may further determine that the rear camera is activated to shoot a video before the camera application generates the UUID.

[0199] As described in the aforementioned software framework, the camera application includes a control module, and the control module can generate a group ID in response to a first operation.

[0200] S103: The camera application sends a group ID to the camera Hal through the camera framework.

[0201] As described above, the application framework layer includes a camera framework. The camera application sends a group ID to the camera Hal through the camera framework. In some embodiments, the camera application may send a group ID to the camera framework, and after receiving the group ID, the camera framework sends the group ID to the camera Hal. The control module of the camera application sends the group ID to the camera framework.

[0202] S104: The camera application sends the capture resolution to the camera Hal through the camera framework.

[0203] Because the electronic device can capture wonderful images in the video while shooting the video in the one-shot mode, the electronic device needs to set the resolution of the captured wonderful images. The capture resolution refers to the resolution set by the electronic device to capture wonderful images from the video.

[0204] In some embodiments, the camera application receives the first operation, and in response to the first operation, the camera application may send the capture resolution to the camera Hal through the camera framework. The camera application may send the capture resolution to the camera framework, and after the camera framework receives the capture resolution, it sends the capture resolution to the camera Hal. The control module of the camera application sends the capture resolution to the camera framework.

[0205] In some other embodiments, the user Figure 2As shown in (a), click the camera application. The camera application receives the click operation and can respond to the click operation by sending the capture resolution to the camera Hal through the camera framework. The camera application can send the capture resolution to the camera framework, and after receiving the capture resolution, the camera framework sends the capture resolution to the camera Hal. Among them, the control module of the camera application sends the capture resolution to the camera framework.

[0206] S105: The camera application sends a first control command to the camera Hal through the camera framework, where the first control command is used to control the camera Hal to run to analyze the video.

[0207] The camera application receives the first operation, and in response to the first operation, controls the camera Hal to analyze the video captured by the camera. Among them, the camera Hal analyzing the video can be understood as: identifying the wonderful moments in the video, the theme, scene and transition of the video, and capturing the images of the wonderful moments. The camera application can issue a first control command to control the camera Hal to analyze the video captured by the camera. In some embodiments, the camera application can send the first control command to the camera framework, and after the camera framework receives the first control command, it sends the first control command to the camera Hal. Among them, the control module of the camera application sends the first control command to the camera framework. In addition, the first control command can control the algorithm recognition module in the camera Hal to analyze the video.

[0208] S106. The camera Hal obtains the video captured by the camera.

[0209] It should be noted that if the user Figure 2 As shown in (a), click the camera application. The mobile phone responds to the operation and controls the camera to capture images. The camera Hal can obtain the image captured by the camera and upload it to the camera application, and the camera application forms and displays a preview interface. Therefore, when the camera Hal receives the first control command, it can continue to obtain the image captured by the camera (which can also be understood as the captured video) through step S106. Of course, the video obtained by the camera Hal can also be uploaded to the camera application.

[0210] S107, the camera Hal sends the video captured by the camera to the Media Recorder.

[0211] The camera Hal obtains the video captured by the camera. In addition to uploading it to the camera application to form a preview interface, it can also be sent to the Media Recorder so that the Media Recorder can record the video.

[0212] S108, Media Recorder records video.

[0213] Media Recorder recording video means: recording the video shot by the camera sent by the camera Hal. Of course, recording video can be understood as: Media Recorder encodes the video shot by the camera and saves the encoded video.

[0214] In addition, when the camera application is started, the storage location of the video captured by the camera can be specified to the Media Recorder. The Media Recorder receives the video sent by the camera Hal and saves the video in the storage location specified by the camera application.

[0215] It should be noted that Fig.10 The example of the execution order of the aforementioned step S102, step S103, step S104, and step S105 is shown, but the execution order of the aforementioned step S102, step S103, step S104, and step S105 is not limited to Fig.10 In some embodiments, step S102 and step S103 may be performed in parallel with step S104 and step S105. In other embodiments, step S103 may be performed in parallel with step S104 and step S105.

[0216] Furthermore, before the camera application executes step S104 and step S105, it may also determine that the one-shot function of the electronic device is in the activated state, as before executing step S102, and in some embodiments, the electronic device further determines that the rear camera is activated to shoot a video. Of course, the camera application may perform a one-shot detection of whether the one-shot function of the electronic device is activated and whether the rear camera is activated to shoot a video, and the camera application executes step S102 and step S103, step S104, and step S105 in parallel based on the result of the one-shot detection.

[0217] S109. The camera Hal recognizes the theme, scene, transition and highlight moment of the video in response to the first control command.

[0218] As in the aforementioned software framework, the camera Hal includes an algorithm recognition module, which can load the AI ​​algorithm. After the algorithm recognition module is loaded with the AI ​​algorithm, the algorithm recognition module can recognize the wonderful moments in the video, the theme, scene and transition of the video, and capture images of the wonderful moments.

[0219] The camera application mentioned in the aforementioned step S105 sends the first control command to the camera Hal through the camera framework, which can be understood as: the camera application sends the first control command to the algorithm recognition module through the camera framework. After the algorithm recognition module receives the first control command, the AI ​​algorithm can be loaded. After the algorithm recognition module has loaded the AI ​​algorithm, it dynamically identifies the subject, scene, transition and wonderful moment of the video (also called video stream) captured by the camera.

[0220] When the camera is shooting video, the algorithm recognition module uses certain strategies to analyze the subject and scene of the image frames captured from the video, as well as identify the wonderful moments. Specifically:

[0221] Regarding thematic analysis of the video: the algorithm recognition module can capture image frames at different moments or continuous image frames from the captured video, and the algorithm recognition module can analyze the captured image frames to determine the subject of the video shot by the camera.

[0222] Regarding the scene and transition analysis of the video: the algorithm recognition module compares two image frames to detect whether a scene change occurs (such as whether a change in the protagonist / environment / scene occurs). If a scene change occurs, the start and end time of the scene, the type of scene to which it belongs, and the type of transition switch are marked. The transition switch type can be understood as indicating the type of scene change, such as a change in the protagonist / environment / scene, where scene changes may include scene changes at the semantic level such as the composition of the image content. Normally, the algorithm recognition module can use the next time it determines the time when the scene changes to determine the stop time of the scene. In addition, the algorithm recognition module can determine the type of scene by analyzing multiple image frames of a scene.

[0223] Regarding the analysis of the highlights of the video: the algorithm recognition module can capture image frames at different moments from the captured video, and determine whether the image frame is a selected image based on factors such as image quality, characters, and actions, that is, whether the moment when the image frame occurs is a highlight moment. In some embodiments, the algorithm recognition module captures multiple image frames from each identified scene and identifies whether the moment when the image frame occurs is a highlight moment.

[0224] Table 2 below gives some examples of scene changes and highlights recognized by the algorithm recognition module.

[0225] Table 2

[0226]

[0227]

[0228] It should be noted that the algorithm recognition module recognizes the subject of the video and the wonderful moments in the video, which can be recorded. The recognized scenes and transitions can be reported to the camera application through the following steps S110 and S111.

[0229] It should also be noted that the camera Hal can also identify the shooting time of the image frames with good quality in the video and save the shooting time of the image frames with good quality.

[0230] S110. When the camera Hal recognizes that a transition in the video ends, the camera maps information of the video segment corresponding to the transition into label data of the video segment.

[0231] During the process of the camera shooting the video, the algorithm recognition module in the camera Hal synchronously analyzes the video being shot. Therefore, when the algorithm recognition module determines that a scene change has occurred in the video, it can determine that a scene (or a transition) of the video has ended. The algorithm recognition module can report the information of the ended scene.

[0232] Because a scene corresponds to a video clip of a video, the scene information is the information of the video clip corresponding to the transition, and generally includes the start and end time of the scene mentioned in the aforementioned step S109, the scene type, and the transition switching type.

[0233] Moreover, the information of the video clip corresponding to the transition can usually be reported in the form of label data. Therefore, the algorithm recognition module can map the information of the video clip corresponding to the transition into the label data of the video clip. The label data of the video clip includes: the start and end time of the scene corresponding to the video clip recorded in the form of a label, the scene type to which it belongs, and the transition switching type. In one example, the start and end time of the scene recorded in the form of a label can be the start and stop time of the scene itself, or the start and stop time of the scene after the format change, or the start and stop time of the scene referred to by the label.

[0234] Exemplarily, the video captured by the camera can be called video 1. The camera captures video 1 at 08:00:00 (absolute time) and ends the capture of the first scene at 08:00:15 (absolute time). The camera Hal recognizes that the scene belongs to a character scene, and the transition type is a change of the protagonist. Based on this, the label data of the video clip corresponding to the first scene includes: the start time is 08:00:00, the stop time is 08:00:15, the scene type is a character scene, and the transition type is a change of the protagonist. In addition, the camera Hal recognizes that there are wonderful moments in the first scene, and the wonderful moments include 08:00:10.

[0235] Furthermore, the camera ends the shooting of the second scene in video 2 at 08:00:35 (absolute time). The second camera Hal recognizes that this scene also belongs to a character scene, and the transition type is a change of the protagonist. Based on this, the label data of the video clip corresponding to the second scene includes: the start time is 08:00:15, the stop time is 08:00:35, the scene type is a character scene, and the transition type is a change of the protagonist. In addition, the camera Hal recognizes that there are also wonderful moments in the second scene, and the wonderful moments include 08:00:22.

[0236] In this example, the first two scenes in video 1 are taken as an example for introduction. The camera Hal can identify other scenes in video 1 through step S110, which will not be described one by one here.

[0237] It should be noted that step S110 may also be a selectively executed step. In some embodiments, after the algorithm recognition module recognizes information about a scene, it may directly report the information to the camera application through the following step S111 without executing step S110.

[0238] S111. The camera Hal sends the tag data of the video clip to the camera application through the camera framework.

[0239] The algorithm recognition module in the camera Hal sends the tag data of the video clip to the camera application through the camera framework. In some embodiments, the algorithm recognition module sends the tag data of the video clip to the camera framework, and after the camera framework receives the tag data of the video clip, it sends the tag data of the video clip to the camera application. After the camera application receives the tag data of the video clip, it can cache the tag data of the video clip.

[0240] As described in the aforementioned software framework, the camera application includes a TAG management module, and the camera Hal sends the tag data of the video clip to the TAG management module in the camera application through the camera framework. After receiving the tag data of the video clip, the TAG management module can cache the tag data of the video clip.

[0241] It should be noted that after the camera Hal obtains the tag data of a video segment corresponding to a transition through step S110, it can report the tag data of the video segment to the camera application through step S111. In addition, when the camera Hal executes step S111 to report the tag data of a video segment to the camera application, the camera Hal can also execute the video analysis process in parallel, that is, analyze the next scene of the video in parallel to obtain the information of the next scene.

[0242] S112. The camera Hal captures a wonderful image from the video. The resolution of the wonderful image is the capture resolution, and the capture moment is the wonderful moment.

[0243] As described in the aforementioned step S109, during the process of the camera shooting a video, the algorithm recognition module in the camera Hal continuously identifies the wonderful moments in the video and records the wonderful moments. In addition, for each wonderful moment identified, the camera Hal can also capture one or more image frames of the wonderful moment from the captured video. The camera Hal can also select a frame of good quality from the captured multiple frames, and the algorithm recognition module will score the wonderfulness of the image frame to obtain a wonderfulness score. In one example, the camera Hal captures 8 frames of the wonderful moment from the captured video, and selects one frame of the image frame for wonderfulness scoring.

[0244] An image frame captured and selected by the camera Hal can be cached in the capture queue. Since the number of image frames saved in the capture queue is limited, in one example, the capture queue can save 5 image frames. Therefore, the algorithm recognition module can dynamically overwrite the image frames cached in the capture queue according to the wonderfulness score of the image frame. For a new image frame captured by the camera Hal, the algorithm recognition module compares its wonderfulness score with the wonderfulness score of the image frame cached in the capture queue. The algorithm recognition module controls the deletion of image frames with a lower wonderfulness score than the new image frame from the capture queue, and controls the caching of new image frames in the capture queue.

[0245] It should be noted that the action of capturing one or more image frames of the wonderful moment from the captured video and the action of selecting an image frame with good quality can be performed by other modules of the camera Hal, which is not included in the Fig. 9 The software framework diagram is shown in the display.

[0246] It should also be noted that the aforementioned step S109 mentions that the wonderful moments in the video will be saved. In some embodiments, the number of wonderful moments saved in a video can be limited. In one example, the number of wonderful moments is limited to 64. Based on this, if the number of saved wonderful moments reaches the limit value, the saved wonderful moments can be dynamically adjusted according to the wonderfulness score of the image frame of the wonderful moment. It is understandable that among the saved wonderful moments, the image frames with low wonderfulness scores should be deleted, and the wonderful moments with high wonderfulness scores should be saved.

[0247] It should also be noted that Fig.10 An execution order of step S110, step S111, and step S112 is depicted, but the execution order of step S110, step S111, and step S112 is not limited thereto. In some embodiments, step S110 and step S111 can be executed in parallel with step S112. In other embodiments, step S110 and step S112 can be executed in parallel.

[0248] S113: The camera application receives a second operation, where the second operation is used to stop shooting the video.

[0249] If the user determines that the video shooting is complete, Figure 3 As shown in (c), the stop control 302 is clicked to input the second operation. The camera application receives the second operation and analyzes the second operation to determine that the operation is used to end the video shooting.

[0250] S114. In response to the second operation, the camera application sends a second control command to the camera Hal through the camera framework, where the second control command is used to control the camera Hal to stop analyzing the video.

[0251] The camera application analyzes the second operation and determines that the operation is used to end the video shooting. In response to the second operation, the camera application issues a second control command. In some embodiments, the camera application sends the second control command to the camera framework, and after the camera framework receives the second control command, it sends the second control command to the camera Hal. In addition, the second control command can be sent by a control module in the camera application.

[0252] After receiving the second control command, the camera Hal may respond to the second control command and stop analyzing the video, that is, stop identifying the wonderful moments in the video, the theme, scene and transition of the video, and may also stop capturing images of the wonderful moments.

[0253] In this embodiment, after the video shooting stops, the camera application sends a second control command to the camera Hal, and the camera Hal stops performing video analysis to reduce the power consumption of the electronic device.

[0254] S115, the camera Hal stops recognizing the subject, scene, transition and highlight moment of the video.

[0255] After receiving the second control command, the camera Hal stops analyzing the video in response to the command, that is, stops identifying the subject, scene, transition and wonderful moment of the video.

[0256] It should also be noted that the camera Hal will continue to obtain the video captured by the video and report it to the camera application to maintain the preview interface. In some embodiments, the camera Hal can also send the obtained video to the Media Recorder, but the Media Recorder will be controlled by the camera application and will not continue to record the video. In other embodiments, the camera Hal may not send the obtained video to the Media Recorder, and the Media Recorder will not receive the video, so it will not continue to record the video.

[0257] S116. Camera Hal writes the group ID into the wonderful image.

[0258] After receiving the second control command, the camera Hal, in addition to stopping the video analysis, can also respond to the second control command and write a group ID in the wonderful image. In some embodiments, the camera Hal writes the group ID in each image frame cached in the capture queue. In some embodiments, the camera Hal writes the group ID in the image frame, which can be understood as: the camera Hal writes the group ID in the image data corresponding to the image frame. The image data corresponding to the image frame can generally be divided into a header data part and an image data part. The header data part includes the attribute information of the image, and the image data part includes the pixel data of the image. The camera Hal writes the group ID in the image data corresponding to the image frame, which can be understood as writing the group ID in the header data part. Exemplarily, the group ID can be written into the extension field of the header data part.

[0259] It should be noted that, as mentioned above, the image frames saved in the capture queue can usually be RAW images. The encoding module in the camera Hal can edit the RAW image to obtain an image in JPEG format or other formats. After the encoding module edits the RAW image, the group ID can also be written into the edited image. Usually, the group ID can be written into the image Exif information, and the image Exif information refers to the image data corresponding to the image frame proposed in the above content.

[0260] In some other embodiments, the camera Hal writes the group ID in the image frame, which can also be understood as: the camera Hal writes the group ID in the name of the image frame, or uses the group ID as the name of the image frame.

[0261] S117. The camera Hal sends the wonderful image to the camera application through the camera post-processing path.

[0262] The camera Hal sends the wonderful image with the group ID written therein to the camera application through the camera post-processing path. In some embodiments, the camera Hal sends the wonderful image with the group ID written therein to the camera post-processing path, and after receiving the wonderful image with the group ID written therein, the camera post-processing path sends the wonderful image with the group ID written therein to the camera application.

[0263] S118. Camera application saves wonderful images.

[0264] The camera application receives the wonderful image with the group ID written thereon and saves it. As described in the aforementioned software framework, the camera application includes a saving module, which is used to save the wonderful image with the group ID written thereon.

[0265] S119, the camera Hal maps the theme and highlights of the video, as well as the group ID, into the tag data of the video.

[0266] As described in the aforementioned step S109, the camera Hal saves the subject and highlights of the identified video. After the camera Hal receives the second control command, the camera Hal may map the subject and highlights of the saved video, as well as the group ID, into the tag data of the video. In some embodiments, the tag data of the video may also include the length of the video. In other embodiments, the tag data of the video may also include the shooting time of the image frame with good quality in the video.

[0267] In some embodiments, the algorithm recognition module maps the theme and highlights of the video, as well as the group ID, into the tag data of the video. Of course, the specific implementation of this step can refer to the content mentioned in the aforementioned step S110, and no further explanation is given here.

[0268] In the example mentioned in step S110, the end time of video 1 identified by camera Hal is: 08:00:60, and the subject of the video is a person, and the wonderful moments include: 08:00:10 and 08:00:22. Therefore, the label data of the video obtained by camera Hal in step S119 includes: the subject of video 1 is a person, the wonderful moments include at least: 08:00:10 and 08:00:22, of course, also includes the group ID of video 1, and the video length is 60 seconds.

[0269] It should be noted that step S120 may also be a selectively executed step. In some embodiments, the camera Hal may directly report the subject and highlights of the video, as well as the group ID, to the camera application through the following step S121 without executing step S120.

[0270] S120. The camera Hal sends the tag data of the video through the camera framework.

[0271] The algorithm recognition module in the camera Hal sends the video tag data to the camera application through the camera framework. In some embodiments, the algorithm recognition module sends the video tag data to the camera framework, and after the camera framework receives the video tag data, it sends the video tag data to the camera application. After the camera application receives the video tag data, it can cache the video tag data.

[0272] As described in the aforementioned software framework, the camera application includes a TAG management module, and the camera Hal sends the tag data of the video to the TAG management module in the camera application through the camera framework. After receiving the tag data of the video, the TAG management module can cache the tag data of the video.

[0273] Table 3 below shows an example of tag data of a video cached by the TAG management module and tag data of the video segment mentioned in the aforementioned step S110.

[0274] Table 3

[0275]

[0276]

[0277] In the example shown in Table 3, the tag data of the video may include: the subject, id, duration and highlights in the first-level tag. Of course, the second-level submenu under the highlights will be included in the tag data of the video. The tag data of the video segment may include: each second-level subtag under the video segment slices in the first-level tag. That is, the startTime, endTime, scene and transition included in the video segment.

[0278] It should also be noted that Fig.10 The execution order of step S116 and step S117, and step S119 and step S120 is shown, but the execution order of step S116 and step S117, and step S119 and step S120 is not limited to this. In some embodiments, step S116 and step S117 can be executed in parallel with step S119 and step S120.

[0279] The foregoing content introduces the method proposed in the first link for shooting a video in a one-record-multiple-receive mode to obtain the shot video and the wonderful images in the video. The following content introduces the process content of the second link with some embodiments.

[0280] Fig.11 Shown Fig. 9 A signaling diagram showing the cooperation of multiple working modules in the software framework of the electronic device to execute the second phase process. Fig. 9 The software framework of the electronic device shown has multiple working modules, which can be used after execution. Fig.10 After step S120 is shown, continue to execute Fig.11 Steps S121 to S137 are shown. Fig.11 The method proposed in the second phase of reviewing the captured videos and the wonderful images in the videos, and generating and saving the selected videos includes the following steps:

[0281] S121. The camera application processes the tag data of the video clip and the tag data of the video to obtain Json data.

[0282] Camera Hal executes step S110 to obtain the tag data of the video segment corresponding to each transition of the video. Moreover, each time the camera Hal recognizes the end of a transition, it can send the tag data of the video segment to the camera application through step S111 of the aforementioned embodiment. Camera Hal can also send the tag data of the video to the camera application through step S120 of the aforementioned embodiment. In this way, the camera application can obtain the tag data of the video and the tag data of the video segment corresponding to each transition. The camera application can process the tag data of the video segment and the tag data of the video to obtain Json data. In some embodiments, the TAG management module of the camera application receives and processes the tag data of the video segment and the tag data of the video to obtain Json data.

[0283] It should be noted that in the tag data of the video clip, the start and end time of the scene corresponding to the video clip is usually an absolute time, which is the real time when the video clip is shot. Based on this, the camera application processing the tag data of the video clip may include: the TAG management module may convert the start and end time of the scene corresponding to the video clip into a relative time, that is, the start time of the first frame image of the video is defined as the zeroth second, and the shooting time of each frame image starting from the second frame image is used as the time interval to obtain the relative shooting time of each frame image starting from the second frame image.

[0284] In the example mentioned in step S110, the camera shoots video 1 from 08:00:00 (absolute time), ends shooting of the first scene at 08:00:15 (absolute time), ends shooting of the second scene at 08:00:35 (absolute time), and ends shooting of video 1 at 08:00:60. In this example, when the TAG management module processes the start and end time of the scene of video segment 1 (i.e., the video segment corresponding to the first scene) in video 1, the start time of video 1 is defined as 0 seconds, and the stop time of video segment 1 corresponding to the first scene is processed as 15 seconds; similarly, the start time of video segment 2 corresponding to the second scene is 15 seconds, and the stop time is 35 seconds. At this time, the starting time of the first scene in relative time form is the 0th second, which can be expressed as 00:00:00, and the stopping time in relative time form is the 15th second, which can be expressed as 00:00:15; the starting time of the second scene in relative time form is the 15th second, which can be expressed as 00:00:15, and the stopping time in relative time form is the 35th second, which can be expressed as 00:00:35.

[0285] It should also be noted that after the TAG management module in the camera application processes the tag data of the video clip using the above content, the TAG management module can further convert the tag data of each processed video clip and the tag data into a format to obtain data in Json format. Of course, the TAG management module can also convert the tag data of each processed video clip and the tag data into data in other formats. In this embodiment, data in Json format is used as an example for explanation, but it does not constitute a limitation on the format conversion mentioned in this step.

[0286] S122. The camera application sends Json data to the Media Recorder.

[0287] The Media Recorder can write data into the video captured by the camera, so the camera application obtains the Json data using the aforementioned step S121 and can send it to the Media Recorder.

[0288] In some embodiments, the TAG management module sends Json data to the Media Recorder.

[0289] S123, Media Recorder writes the Json data into the video header of the video.

[0290] After receiving the Json data, the Media Recorder writes it into the video header of the video. In this embodiment, the video refers to a video file. Usually, a video file includes a video header and video content. The video header is provided with a custom extended information field, and the Json data can be written into the custom extended information field in the video header.

[0291] It should be noted that, as described in step S115 in the above embodiment, after the user inputs the second operation to stop shooting the video, the Media Recorder may be controlled by the camera application and stop recording the video, or the Media Recorder no longer receives the video sent by the camera Hal and stops recording the video. In addition, the Media Recorder may also write Json data in the video header of the recorded video.

[0292] In some embodiments, after the user inputs the second operation to stop shooting the video, the Media Recorder may first receive the Json data in step S122, and write the Json data into the video header of the recorded video in step S123. Thereafter, the Media Recorder stops recording the video and performs conventional processing on the recorded video with the Json data, such as packaging and saving.

[0293] S124: The camera application sends the attribute information and storage path of the video to the Media Provider.

[0294] The video captured by the camera will be saved in the internal memory of the electronic device. In order to facilitate the management of the videos saved in the internal memory, the Media Provider can establish a management library for the relevant information of the videos stored in the internal memory. Based on this, the camera application can send the attribute information and storage path of the video captured by the camera to the Media Provider.

[0295] After receiving the attribute information and storage path of the video, the Media Provider may store it in the management library. The attribute information of the video is used to indicate the basic attributes of the video, and may include: the generation time of the video, the codec format, the resolution information, etc. In some embodiments, the basic attributes of the video may also include an indication tag, which is used to indicate that the video was shot by the electronic device using a one-shot multiple-record mode.

[0296] S125. Media Provider triggers Media Library to synchronize data saved by Media Provider.

[0297] Media Library can obtain the data stored in the management library of Media Provider. Therefore, Media Provider triggers Media Library to synchronize the data saved by Media Provider.

[0298] The Media Provider can send its stored data to the Media Library. In some embodiments, the Media Library can set a Uniform Resource Identifier (URI) and use the URI to monitor the data of the Media Provider. If the data stored by the Media Provider changes, the Media Provider sends the data to the Media Library.

[0299] Of course, the Media Library can also obtain the data stored by the Media Provider. In some embodiments, the Media Library uses the URI to monitor the data of the Media Provider. If the data stored by the Media Provider changes, the Media Library obtains the data stored by the Media Provider.

[0300] S126. Media Library obtains the video using the storage path of the video, and obtains the group ID and video type of the video from the video header of the video.

[0301] The data types stored in the management library in the Media Provider are limited, and some data cannot be stored in the management library. Therefore, data that cannot be stored in the Media Provider will be stored in the Media Library. In some embodiments, the Media Library stores the group ID and video type of the video.

[0302] Media Library can query videos according to the video storage path and obtain the queried videos. It can obtain tag data in the video header of the video, parse the tag data, and obtain the group ID and video type of the video.

[0303] It should also be noted that the data stored in the Media Library can be provided to the gallery application.

[0304] S127. Media Library generates a correspondence between the video and the wonderful image using the group ID.

[0305] The Media Library traverses the wonderful images stored in the electronic device, parses the wonderful images, such as parsing the image Exif information of the wonderful images, and obtains the group ID written in the wonderful images. In addition, the Media Library traverses the videos stored in the electronic device, parses the custom extended information field in the video header of the video, and obtains the group ID in the custom extended information field. The Media Library uses the group ID to generate a correspondence between the video and the wonderful image. Among them, the Media Library establishes a correspondence between the video and the wonderful image with the same group ID.

[0306] S128. The gallery application receives a third operation, where the third operation is used to display videos and wonderful images.

[0307] User Figure 6 As shown in (a), the user clicks the control 501 showing the image captured last time to enter the third operation. Figure 6 In the photo display interface of the gallery shown in (b), the third operation is entered by clicking the cover thumbnail of the video 502. The gallery application analyzes the third operation and determines that the operation is for displaying videos and wonderful images.

[0308] Furthermore, the gallery application analyzes the third operation and may further determine that the third operation specifies a video to be displayed.

[0309] S129: The gallery application obtains the group ID and storage path of the videos stored in the Media Library.

[0310] In response to the third operation, the gallery application traverses the information of the videos stored in the Media Library, determines the information of the video specified by the third operation, and obtains the group ID and storage path of the video.

[0311] S130: The gallery application obtains the video by using the storage path of the video, and obtains the wonderful image by using the correspondence between the video and the wonderful image.

[0312] The storage path of the video is used to indicate the storage address of the video. Therefore, the gallery application can use the storage path of the video to query and obtain the video. The gallery application also uses the corresponding relationship between the video and the wonderful image to obtain the wonderful image corresponding to the video.

[0313] S131. The gallery application associates and displays the video and the wonderful images according to the group ID.

[0314] The video captured by the camera using the one-record-multiple-receive module and the wonderful images in the video are set with the same group ID. Therefore, when the gallery application obtains the video and the wonderful images corresponding to the video, it can display the video and the wonderful images in association according to the group ID set.

[0315] The gallery application can display the video and wonderful images in association with each other according to the group ID. Figure 6 As shown in (c) or (d).

[0316] S132. The gallery application receives a fourth operation, where the fourth operation is used to generate a selected video.

[0317] User Figure 7 As shown in (a), the fourth operation is input by clicking on the control 601. The gallery application analyzes the fourth operation and determines that the operation is used to generate a selected video.

[0318] S133: The gallery application sends the storage path of the video to the video editing application.

[0319] In response to the fourth operation, the gallery application jumps to the video editing application to display the interface of the video editing application. In the process of jumping from the camera application to the video editing application, the camera application sends the storage path of the video information to the video editing application.

[0320] The gallery application uses the storage path of the video information as a write parameter of the interface, calls the interface of the video editing application, realizes the jump from the camera application to the video editing, and sends the storage path of the video information to the video editing application.

[0321] S134: The video editing application obtains the video using the storage path of the video.

[0322] The video editing application obtains the storage path of the video, and obtains the video using the storage path of the video.

[0323] S135. The video editing application utilizes the video to obtain a selected video.

[0324] As described in the above step S123, the video header of the video obtained by the video editing application includes tag data in the Json format. Based on this, the video editing application can obtain the tag data from the video header.

[0325] The video editing application parses the tag data to obtain the tag data of the video segments, and segments the video according to the start and end time of each video segment. The video editing application selects high-value segments that can be used for filming from the video to generate a set of candidate segments, and deletes the unselected segments. In some embodiments, the video editing application can use the highlights in the tag data to select high-value segments.

[0326] The video editing application performs a deduplication process on the video segments in the candidate segment set to remove the same or similar video segments.

[0327] The video editing application can also select the corresponding style based on the theme tags in the tag data. Each theme tag corresponds to at least one style. Each style will match a background music by default, and at the same time, each style will correspond to at least one filter, sticker, transition, special effect and other video editing information. After the video editing application determines the background music, it determines the transition position between video clips to the rhythm of the background music. In addition, the video editing application can select the transition type based on the theme tag or user operation and camera movement tag, and select and determine the corresponding transition type from the transition resource library.

[0328] The video editing application performs segment splicing layout on the video clips. Among them, the arrangement strategy of the splicing layout can be, for example, selecting high-value clips and arranging them into a film in chronological order, and adding transitions between every two video clips. The shot constraint can be: the film includes 2 to 5 clips (at least 3 as much as possible), and the duration of each clip is 2 seconds to 6 seconds. And the background music card point constraint can be: the segment splicing transition position matches the background music rhythm point as much as possible. The duration constraint of the film can be: the duration of the film is between 10 seconds and 15 seconds.

[0329] The video editing application uses the video processed by the segment splicing layout as the intermediate video and adds other beautification strategies to it. The other beautification strategies include adding other beautification effects such as filters, stickers, special effects, etc. The video editing application confirms that the addition is completed as other beautification strategies, and the selected video can be obtained.

[0330] It should be noted that if the video clips in the candidate clip set are insufficient to generate a selected video, the video editing application may select video clips corresponding to scenes with high matching degrees according to the matching degrees of scene tags and subject tags to generate a selected video.

[0331] It should also be noted that if the label data does not include highlights, the video editing application can use the shooting time of the image frame with good quality, select the video clip corresponding to the shooting time, and generate a selected video.

[0332] S136: The video editing application receives a fifth operation, where the fifth operation is used to save the selected video.

[0333] User Figure 8 As shown in (a), the save control 604 is clicked to input the fifth operation. The gallery application analyzes the fifth operation and determines that the operation is for saving the selected video.

[0334] S137. The video editing application saves the selected video in response to the fifth operation.

[0335] Among them, the video editing application saves the selected video to the internal storage, which is usually the same as the storage location of the video, and is also the storage location specified by the camera application.

[0336] It should also be noted that in some embodiments, before the video editing application saves the selected video, the video editing application may also write a group ID in the selected video, specifically, the group ID may be written in the video header of the selected video. The group ID may also be written into a custom extended information field in the video header.

[0337] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0338] The computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0339] Another embodiment of the present application further provides a computer program product including instructions. When the computer program product is run on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

Claims

1. A video processing method, characterized in that: Applied to electronic equipment, the video processing method comprises: In response to a first operation, a first interface is displayed and a first video is started to be shot, wherein the first interface is a shooting interface of a camera application, and the first interface includes a first control, and the first control is used to stop shooting the first video; During the shooting process of the first video, obtaining the first video; Identify the theme, transitions and highlights of the first video, and obtain tag data of each transition of the first video and tag data of the first video; the transition is used to indicate the conversion of scenes in the first video, and the tag data of each transition includes: information of the video clip corresponding to the transition; the tag data of the first video includes the theme tag, highlights and group ID of the first video; Using the N wonderful moments, obtain Y wonderful images from the first video, where N is an integer greater than or equal to Y; In response to a second operation, the tag data of each transition of the first video and the tag data of the first video are saved to the video header of the first video, and the Y wonderful images are saved, where the wonderful images include the group ID of the first video, and the second operation is a trigger operation for the first control; In response to a third operation, acquiring the first video and the Y wonderful images corresponding to the group ID of the first video, wherein the third operation is a browsing operation of the first video; Displaying a browsing interface of the first video, wherein the browsing interface of the first video includes: thumbnails of the first video and the Y wonderful images; The video editing application of the electronic device obtains the first video in response to the generation operation of the selected video; The video editing application obtains tag data of each transition of the first video and tag data of the first video from a video header of the first video; The video editing application determines a style corresponding to the theme tag of the first video, the style matches the corresponding background music, and the corresponding video editing information, the video editing information is used to beautify the display effect of the video; The video editing application parses the label data to obtain label data of the video segments, and segments the first video according to the start and end time of each video segment to obtain multiple video segments, and selects high-value segments from the multiple video segments; The video editing application splices the high-value clips in the order of the shooting time of the high-value clips, combines the background music with the spliced ​​high-value videos, and adds a transition video between the two high-value clips to obtain a second video; The video editing application adds a beautification strategy to the second video to obtain a selected video, wherein the beautification strategy includes elements of the video editing information for beautifying the video.

2. The video processing method according to claim 1, characterized in that: The method of obtaining Y wonderful images from the first video by using the N wonderful moments includes: Capturing N images of the wonderful moments from the first video; Among the N images of the wonderful moment, Y images with high wonderfulness scores are selected as the wonderful images.

3. The video processing method according to claim 1, characterized in that: The method of obtaining Y wonderful images from the first video by using the N wonderful moments includes: The camera Hal uses the N wonderful moments to obtain Y wonderful images from the first video.

4. The video processing method according to claim 1, characterized in that: In response to the third operation, acquiring the first video and the Y wonderful images corresponding to the group ID of the first video; The browsing interface for displaying the first video includes: In response to the third operation, the gallery application obtains the first video and the Y wonderful images corresponding to the group ID of the first video; The gallery application displays a browsing interface of the first video.

5. The video processing method according to claim 4, characterized in that: The gallery application obtains the first video and the Y wonderful images corresponding to the group ID of the first video in response to the third operation, including: In response to the third operation, the gallery application obtains the storage path of the first video from the media information library; The gallery application obtains the first video by using the storage path of the first video; The gallery application parses the first video, obtains the group ID of the first video, and acquires the Y wonderful images corresponding to the group ID of the first video.

6. The video processing method according to claim 1, characterized in that: The identifying the theme, transitions, and highlights of the first video, and obtaining label data of each transition of the first video and label data of the first video, includes: Using the images in the first video, identifying the theme, transitions, and highlights of the first video, and obtaining a theme tag of the first video; When recognizing that a transition in the first video ends, generating label data of the transition using information of a video segment corresponding to the transition; When the shooting of the first video is finished, label data of the first video is generated using the subject label, highlight moments and group ID of the first video.

7. The video processing method according to claim 1, characterized in that: The step of storing the tag data of each transition of the first video and the tag data of the first video in the video header of the first video includes: The media recorder receives the tag data of each transition sent by the camera application and the subject tag of the first video; The media recorder writes the tag data of each transition and the subject tag of the first video into the video header of the first video.

8. The video processing method according to claim 1, characterized in that: The video editing application of the electronic device obtains the first video in response to the generation operation of the selected video, including: The video editing application receives the storage path of the first video sent by the gallery application in response to the operation of generating the selected video; The video editing application obtains the first video by using the storage path of the first video.

9. The video processing method according to claim 8, characterized in that: Also includes: In response to a save operation of the selected video, the selected video is saved.

10. An electronic device, characterized in that: include: One or more processors, memory, cameras, and displays; The memory, the camera and the display screen are coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the video processing method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the video processing method according to any one of claims 1 to 9.

Images