Video editing method and related device

Abstract

The invention provides a video editing method and a related device. In the method, the electronic equipment can add key frames to a video and a plurality of materials, can edit one or more frames in a plurality of material tracks of the video, and can generate parameters of a plurality of pictures between any two key frames in the video. Therefore, in the process of playing the edited video, the electronic equipment can observe that the video and the plurality of tracks are relatively smooth and gradually generate changes such as rotation, zooming, position movement and opacity change, so that the impression of a user is good.

Description

Technical Field

[0001] The present application relates to the field of video editing technology, and in particular to a video editing method and related devices. Background Art

[0002] Currently, electronic devices can edit videos and change parameter information of multiple frames in a video by adding key frames. In addition, electronic devices can add multiple tracks of content to a video, such as picture-in-picture, text, stickers, etc. How to achieve smooth editing of multi-track videos is an urgent problem to be solved in this field. Summary of the invention

[0003] The present application provides a video editing method and related devices, which can add various material tracks and add key frames to various material tracks, and can also edit one or more frames in the material track, and can also control the parameters of multiple pictures between any two key frames in the track.

[0004] In a first aspect, the present application provides a video editing method, characterized in that the method comprises:

[0005] A first user interface is displayed, the first user interface comprising: a preview area, a first track, a second track and a pointer at a fixed position, the first track displays multiple frames of a first video, the second track displays multiple frames of a first material, the preview area displays a first frame of the first track pointed to by the pointer and a second frame of the second track pointed to by the pointer, and each frame of the first material is presented as a first effect. An operation of selecting the second track is received. An operation of adding a keyframe is received. A left swipe operation on the second track is received, the first track and the second track are moved to the left, and a third frame of the first track pointed to by the pointer after being moved to the left and a fourth frame of the second track pointed to after being moved to the left are displayed in the preview area. An editing operation acting on the fourth frame in the preview area is received, and the effect of the fourth frame in the preview area is adjusted to the second effect. A right swipe operation on the second track is received, the first track and the second track are moved to the right, and a fifth frame of the first track pointed to by the pointer after being moved to the right and a sixth frame of the second track pointed to after being moved to the right are displayed in the preview area. Upon receiving a play operation, the first video is played starting from the fifth frame in the preview area, and the second material is played starting from the sixth frame, wherein the second material includes each frame of the first material, and in the second material, multiple frames before the second frame are presented as the first effect, multiple frames from the second frame to the fourth frame gradually transition from the first effect to the second effect, and multiple frames after the fourth frame are presented as the second effect.

[0006] By implementing the above content, the electronic device can add various material tracks, add key frames to various material tracks, edit one or more frames in the material track, and generate parameters of multiple pictures between any two key frames in the track. In this way, when the electronic device plays the edited video, it can be observed that the changes between different tracks are independent, and the material track is relatively smooth and gradually rotates, scales, moves the display position, changes the opacity, etc., so that the user has a good viewing experience.

[0007] In conjunction with the first aspect, in some implementations, the second track is a picture-in-picture track, the first material is a video containing multiple frames, and in the preview area, the first material is displayed in the first video in a picture-in-picture format. Or the second track is a text track, the first material is a video containing multiple frames of text. Or the second track is a sticker track, the first material is a video containing multiple frames of stickers.

[0008] In some implementations, the first material may also include pictures of more types of materials.

[0009] In conjunction with the first aspect, in some implementations, the first effect includes displaying at a first display position, the editing operation is used to adjust the screen from the first display position to a second display position, and the second effect includes displaying at the second display position. And / or the first effect includes displaying at a first size, the editing operation is used to adjust the screen from the first size to a second size, and the second effect includes displaying at the second size. And / or the first effect includes displaying at a first display angle, the editing operation is used to adjust the screen from the first display angle to the second display angle, and the second effect includes displaying at the second display angle. And / or the first effect includes displaying at a first opacity, the editing operation is used to adjust the screen from the first opacity to the second opacity, and the second effect includes displaying at the second opacity.

[0010] In conjunction with the first aspect, in some implementations, after receiving the operation of adding a keyframe, the method further includes: displaying a first mark at the position of the second track pointed to by the pointer. After receiving the editing operation acting on the fourth picture in the preview area, the method further includes: displaying a second mark at the position of the second track pointed to by the pointer after being moved to the left.

[0011] In combination with the first aspect, in some implementations, after receiving the operation of adding a key frame, the method further includes: acquiring a parameter of the second picture, wherein the parameter of the second picture is used to indicate the first effect.

[0012] After receiving the editing operation acting on the fourth picture in the preview area, the method further includes: acquiring a parameter of the fifth picture, where the parameter of the fifth picture is used to indicate the second effect.

[0013] In conjunction with the first aspect, in some implementations, before displaying the first user interface, the method further includes: displaying a second user interface, the second user interface including: the preview area, the first track, and the pointer with a fixed position. An operation of adding the second track is received.

[0014] In conjunction with the first aspect, in some implementations, after the first video is played from the sixth screen and the second material is played from the seventh screen in the preview area, the method further includes: receiving a user operation for exporting a video. A video including the first video and the second material is stored.

[0015] In combination with the first aspect, in some implementations, in the preview area, the picture of the first material is overlaid on the picture of the first video, or the picture of the first material and the picture of the first video are displayed in split screen.

[0016] In combination with the first aspect, in some implementations, the first track displays all or part of the first video.

[0017] In conjunction with the first aspect, in some implementations, the method further includes:

[0018] The start time of the first track is consistent with the start time of the second track.

[0019] In combination with the first aspect, in some implementations, after receiving an operation of selecting the second track, the method further includes: displaying a first control in the first user interface, and the operation of adding a keyframe includes an operation acting on the first control.

[0020] In combination with the first aspect, in some implementations, after the first video is played from the fifth screen and the second material is played from the sixth screen in the preview area, the method further includes: receiving an operation of pausing playback, and displaying the seventh screen of the first track pointed to by the pointer and the eighth screen of the second track pointed to by the pointer in the preview area. Receiving an editing operation acting on the eighth screen in the preview area, adjusting the effect of the eighth interface in the preview area to the third effect, and the eighth screen is located between the second interface and the fourth screen. Receiving a right swipe operation on the second track, moving the first track and the second track to the right, and displaying the ninth screen of the first track pointed to by the pointer after the right shift and the tenth screen of the second track pointed to by the pointer in the preview area. After receiving a play operation, the first video is played from the ninth screen in the preview area, and the third material is played from the tenth screen. The third material includes screens of the first material arranged in sequence. In the third material, multiple screens before the second screen are presented as the first effect, multiple screens from the second screen to the eighth screen gradually transition from the first effect to the third effect, multiple screens from the eighth screen to the fourth screen gradually transition from the third effect to the second effect, and multiple screens after the fourth screen are presented as the second effect.

[0021] In combination with the first aspect, in some implementations, after receiving the operation of selecting the second track, the method also includes: displaying a left handle on the left edge of the second track, and displaying a right handle on the right edge of the second track, the left handle being used to adjust the start time of the second track, and the right handle being used to adjust the end time of the second track.

[0022] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method of the first aspect or any implementation of the first aspect.

[0023] In a third aspect, an embodiment of the present application provides a chip, comprising one or more processors. When the one or more processors execute computer instructions, the electronic device executes the method of the first aspect or any implementation of the first aspect.

[0024] In a fourth aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the above-mentioned first aspect or any implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1A-Figure 1D A schematic diagram of a user interface for an electronic device to enter a video editing interface provided by an embodiment of the present application;

[0026] Figure 2A-2D A schematic diagram of a user interface for rotating a video by an electronic device provided in an embodiment of the present application;

[0027] Figure 2E-2H A schematic diagram of a user interface for an electronic device to play a rotated video provided by an embodiment of the present application;

[0028] Figure 2I-2J A schematic diagram of a user interface for a video provided by an electronic device according to an embodiment of the present application for performing video zooming processing;

[0029] Figure 3A-Figure 3F A schematic diagram of a user interface for changing the opacity of a picture-in-picture track in an electronic device provided in an embodiment of the present application;

[0030] Figure 3G-3J A schematic diagram of a user interface for an electronic device to play an edited video provided in an embodiment of the present application;

[0031] Figure 4A-4F A schematic diagram of a user interface for changing the position of a text track of an electronic device provided in an embodiment of the present application;

[0032] Figure 4G-4J A schematic diagram of a user interface for an electronic device to play an edited video provided by an embodiment of the present application;

[0033] Figure 5 A flowchart of a video editing method provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of a calculation rule for obtaining parameters of multiple pictures provided in an embodiment of the present application;

[0035] Figure 7 A flowchart of another video editing method provided in an embodiment of the present application;

[0036] Figure 8 An OS interaction diagram corresponding to a video editing method provided in an embodiment of the present application;

[0037] Fig. 9 An OS interaction diagram corresponding to another video editing method provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0039] Fig.11A software structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0041] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0042] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0043] First, two related terms involved in the embodiments of the present application are introduced:

[0044] Keyframe: It is a computer animation term. A keyframe can refer to a frame where a key action is located in the motion change of a character or object. Among them, a frame is a single image with the smallest unit of influence in an animation, which is equivalent to each frame of a movie film. In an embodiment of the present application, a keyframe can be represented by a mark added to a certain position in a material track in a user interface displayed by a video editing application of an electronic device. It is equivalent to that the frame where the mark is located can be considered as a keyframe. Adding a keyframe means marking a frame as a keyframe.

[0045] Track: It can be used to place materials and edit the materials. The length of the track can be used to represent the duration of the material, and each position in the track can be used to represent a frame of the material. Placing different materials can generate different tracks. For example, the main video track can display multiple frames of the main video, and the text track can display text images, etc.

[0046] With the development of electronic devices, electronic devices can meet users' various needs for video editing. For example, an electronic device can set any two frames in a video as key frames, and generate a key frame animation between the two frames, so that the picture in the video can change the display position, size, rotation angle, opacity, etc., so that the video can be played with a richer effect. For another example, the electronic device can also add more material tracks, such as picture-in-picture, text, stickers and other material tracks, so that when the electronic device plays the video, it can not only display the content of the video itself, but also display the added picture-in-picture, text and stickers, etc., so that the video can show more elements and the user has a good viewing experience. In addition, the electronic device can also move the position, change the size, rotate the angle, and change the opacity of the picture in the added track. The above-mentioned generation of key frame animation can be realized by adding key frames to the electronic device. In a possible implementation method, adding a key frame to a frame in the video can be understood as marking a frame of the video. Regarding how to generate key frame animation according to key frames, reference can be made to the introduction of subsequent specific embodiments, which will not be described here.

[0047] The following specifically introduces how the electronic device enters the video editing interface in conjunction with the user interface provided by the electronic device.

[0048] Figure 1A-Figure 1D The user interface of the electronic device provided in the embodiment of the present application for entering the video editing interface is exemplified.

[0049] like Figure 1AAs shown, the user interface 20 is the main interface provided by the electronic device. The user interface 20 may include, but is not limited to: time, date, and various application icons, such as a gallery icon 201. The electronic device may enter the gallery application in response to a user's click operation on the gallery icon 201. In the embodiment of the present application, the method for entering the gallery application is not limited, and may also include entering the gallery application in response to a voice command or a gesture command.

[0050] like Figure 1B As shown, the user interface 21 is a user interface provided by the gallery application in the electronic device. The user interface 21 may include, but is not limited to: a selection bar 211, a photo date 212, a photo control 213, a video control 214, and the like. Among them, the selection bar 211 may include: a photo option 211A, an album option 211B, and a recommendation option 211C. Exemplarily, the user interface 21 may be a user interface in which the photo option 211A is selected in the selection bar 211. Exemplarily, the photo date 212 may be used to indicate the date on which the photo was taken or the date on which the photo was saved. Exemplarily, the photo displayed by the photo control 213 may be taken or saved by the electronic device "today". The video control 214 displays a video time length "00:10", which indicates that the duration of the video displayed by the video control 214 is 10s. The electronic device may enter the full-screen display interface of the video control 214 in response to a click operation of the user on the video control 214.

[0051] like Figure 1CAs shown, the user interface 22 is a user interface for the electronic device to display the video displayed in the video control 214 in full screen. The user interface 22 may include, but is not limited to: a screen direction control 221, a share control 222, an edit control 223, a set favorite control 224, a delete control 225, a more control 226, a video display area 227, a play control 228, and a video track 229. Among them, the electronic device can respond to a click operation on the screen direction control 221 to lock the screen direction or unlock the screen direction. The electronic device can respond to a click operation on the share control 222 to share the video displayed by the video control 214 with a friend. The electronic device can respond to a click operation on the set favorite control 224 to classify the video displayed by the video control 214 into the "My Favorite" type. The electronic device can respond to a click operation on the delete control 225 to delete the video displayed by the video control 214. The electronic device can respond to a click operation on the more control 226 to display more option controls for the video displayed by the video control 214. The electronic device can respond to a click operation on any position of the video display area 227 and hide other elements except the video display area 227. The electronic device can respond to a click operation on the play control 228 and play the video displayed by the video control 214. The video track 229 includes a pointer 229A and a draggable area 229B. The electronic device can respond to a sliding operation on the draggable area 229B and display a frame of the video track 229 pointed to by the pointer 229A. The electronic device can respond to a click operation on the edit control 223 and display the user interface 23.

[0052] like Figure 1DAs shown, the user interface 23 is a user interface provided by a video editing application in an electronic device. The user interface 23 may include, but is not limited to: a video preview interface 231, a video track 232, a pointer 233, a play control 234, an effect option bar 235, a track option bar 236, a play time 237, a resolution selection control 238, an export control 239, and the like. Among them, the video track 232 displays multiple frames of the video. In a possible implementation, if the video is too long, the video track 232 may only display the pictures in part of the video, and the pictures in the rest of the video are hidden. The picture displayed in the video preview interface 231 is a frame of the video track 229 pointed to by the pointer 233. The electronic device can edit the video displayed in the video preview interface 231 in the user interface 23. The effect option bar 235 may include, but is not limited to: split, intercept, volume condition, speed change, delete, crop, single frame export, filter, adjustment parameters (such as color, brightness, contrast, etc.), opacity 235A, cut picture-in-picture, change, and other options. The track option bar 236 may include but is not limited to: template, clip, filter, music, text 236A, sticker 236B, special effect, picture-in-picture 236C, frame, background, etc. The play time 237 is exemplarily represented as "00:00 / 00:10", which means that the position pointed by the current pointer 233 is located at the 0th second frame in a video with a total duration of 10 seconds.

[0053] Based on the user interface introduced above, the user interface for editing and processing videos and playing edited videos provided by the electronic device provided in the embodiment of the present application will be specifically introduced in combination with different scenarios.

[0054] Scenario 1: A scenario for editing a video and playing the edited video.

[0055] Figure 2A-2D The user interface of an electronic device performing a rotation process on a video by adding key frames is exemplarily shown.

[0056] like Figure 2A As shown, the electronic device can display a user interface 30 provided by a video editing application. In response to a click operation on a key frame adding control 301, the electronic device can add a key frame 311 to the Ath frame of the video track 303 pointed to by the pointer 302, that is, the frame displayed in the video preview area 304, and display the following. Figure 2B The user interface 31 is shown.

[0057] Exemplarily, the video track 312 in the user interface 31 includes a key frame 311 added for the Ath frame. The electronic device can respond to pressing the key frame 311 for the video track 312. Figure 2B Slide in the direction of the arrow shown in the Figure 2C The user interface 32 is shown. In the embodiment of the present application, the above sliding operation on the video track 312 is only an exemplary description, and more other operations may be included to change the screen pointed by the pointer, which is not limited to this.

[0058] For example, the pointer 321 in the user interface 32 points to the Bth frame in the video track 322, and the video preview area 323 displays the Bth frame. The electronic device can respond to the operation of rotating the video preview area 323 and display the following information: Figure 2D The user interface 33 shown in FIG. 10 shows an example of a rotation operation. For example, the operation of rotating the screen may include: pressing two fingers to rotate the screen respectively. Figure 2C In the embodiment of the present application, the above operation of rotating the screen is only for illustrative purposes, and may also include more other operations, such as voice commands, gesture commands, long-press mouse movement operations, etc., which are not limited to this.

[0059] Exemplarily, the video track 331 in the user interface 33 includes a key frame 332 added for the B-th frame. It is worth noting that the above-mentioned operation of rotating the picture in the video preview area 323 actually changes the parameters of the B-th frame, and the operation of rotating the picture is equivalent to adding a key frame for the B-th frame. The B-th frame displayed in the video preview area 333 is rotated 90 degrees counterclockwise around the midpoint of the picture relative to the B-th frame displayed in the video preview area 323.

[0060] From the above editing process, it can be seen that the electronic device rotates the video and adds two key frames to the video. Therefore, in addition to the B frame in the video being rotated, the images between the A frame and the B frame in the video are also rotated.

[0061] Figure 2E-2H The user interface of an electronic device playing a rotated video is exemplarily shown.

[0062] Electronic devices can respond to Figure 2D In the video track 331 Figure 2B Slide in the opposite direction of the arrow shown in the figure to display Figure 2E The user interface 34 shown. The pointer 341 in the user interface 34 points to the A-th frame in the video track 342. The electronic device can respond to a click operation on the play control 343 and play the rotated video in the video preview area 344.

[0063] When the rotated video is played to the Cth frame, the electronic device can display the following Figure 2FThe user interface 35 shown. The pointer 351 in the user interface 35 points to the C-th frame in the video track 352, and the C-th frame is displayed in the video preview area 353. The C-th frame displayed in the video preview area 353 is rotated a degrees counterclockwise relative to the C-th frame in the unrotated video. Where a is greater than 0.

[0064] When the rotated video is played to the Dth frame, the electronic device may display the following Figure 2G The user interface 36 shown. The pointer 361 in the user interface 36 points to the D-th frame in the video track 362, and the D-th frame is displayed in the video preview area 363. The D-th frame displayed in the video preview area 363 is rotated b degrees counterclockwise around the midpoint of the frame relative to the D-th frame in the unrotated video. Where b is greater than a.

[0065] When the rotated video is played to the Bth frame, the electronic device may display the following Figure 2H The user interface 37 shown. The pointer 371 in the user interface 37 points to the B-th frame in the video track 372, and the B-th frame is displayed in the video preview area 373. The B-th frame displayed in the video preview area 373 is rotated c degrees counterclockwise around the midpoint of the frame relative to the B-th frame in the unrotated video. Where c is greater than b.

[0066] When the rotated video plays to the frame after the Bth frame, the frame displayed in the video preview area is rotated counterclockwise by c degrees around the midpoint of the frame relative to the frame in the unrotated video.

[0067] From the above process of playing the video after editing, it can be known that the electronic device performs corresponding processing on the video between two key frames to generate a rotation animation for the video. In addition, during the process of playing the video after editing, the video can be rotated smoothly and gradually, rather than the screen suddenly rotating greatly when the video is played to the second key frame, which is good for the user.

[0068] In the embodiment of the present application, in addition to the above-mentioned rotation processing for the video, the video can also be scaled, repositioned, and transparency changed. The electronic device can refer to the above-mentioned user interface for rotating the video, which is not repeated here.

[0069] The following is a detailed introduction to the user interface for scaling the video.

[0070] Figure 2I-2J The user interface of an electronic device performing zooming processing on a video is exemplarily shown.

[0071] The electronic device can respond to the button for the video track 312. Figure 2B Slide in the direction of the arrow shown in the Fig.2I User interface 38 is shown.

[0072] For example, the pointer 381 in the user interface 38 points to the Bth frame in the video track 382, ​​and the video preview area 383 displays the Bth frame. The electronic device can respond to the operation of zooming out the video preview area 383 and display the following information: Figure 2J The user interface 39 shown in FIG. 10 shows an example of a zoom-out operation. For example, the zoom-out operation may include: pressing two fingers to Fig.2I In the embodiment of the present application, the above operation of zooming out the screen is only for illustrative purposes, and may also include more other operations, such as voice commands, gesture commands, long-press mouse movement operations, etc., which are not limited to this.

[0073] Exemplarily, the video track 391 in the user interface 39 includes a key frame 392 added for the B-th frame. It is worth noting that the above-mentioned operation of zooming out the video preview area 393 actually changes the parameters of the B-th frame, and the operation of zooming out is equivalent to adding a key frame for the B-th frame. The B-th frame displayed in the video preview area 393 is reduced by half relative to the B-th frame displayed in the video preview area 383.

[0074] Optionally, the electronic device may also enlarge the picture in the video, which is not limited to this.

[0075] From the above editing process, it can be seen that the electronic device has reduced the size of the video and added two key frames to the video. Therefore, in addition to the B frame in the video being reduced, the images between the A frame and the B frame in the video are also reduced.

[0076] Similarly, the process of playing the reduced video by the electronic device can refer to the above Figure 2E-2H The user interface is not described here. The electronic device processes the video between the two key frames accordingly and generates a zoom-out animation for the video. And in the process of playing the video after the editing process, the video can be relatively smooth and gradually reduced or enlarged, instead of the screen suddenly being greatly reduced or enlarged when the video is played to the second key frame, which is good for the user's viewing experience.

[0077] In some implementations, when a video includes multiple material tracks, the video, that is, the video without any processing or the video after only rotating, scaling, changing the position, changing the opacity, and the like, can be called a video track. This naming is to distinguish it from other material tracks in the subsequent introduction.

[0078] Scenario 2: Adding a picture-in-picture track based on the video track, editing the picture-in-picture track, and playing the edited video.

[0079] Playing the video after editing may refer to playing the video including the picture-in-picture track after editing and the video track after rotation.

[0080] Figure 3A-Figure 3F The user interface of an electronic device adding a picture-in-picture track and changing the opacity of the picture-in-picture track is exemplarily shown.

[0081] Electronic devices can respond to Figure 2D In the video track 331 Figure 2B Slide in the opposite direction of the arrow shown in the Figure 3A The electronic device can respond to a click operation on the picture-in-picture option 401A in the track option bar 401 and select the picture-in-picture that the user wants to add. The picture-in-picture can refer to a photo or a video. Then the electronic device can display the picture-in-picture as shown in FIG. Figure 3B User interface 41 is shown.

[0082] Exemplarily, the video preview area 411 in the user interface 41 includes two parts: a video screen 411A and a picture-in-picture screen 411B. Among them, the electronic device defaults that the added picture-in-picture screen is displayed in the center and overlaid on the video screen. Optionally, the electronic device can also set the video screen to cover the picture-in-picture screen, which is not limited. The user interface 41 may include: a video track 412 and a picture-in-picture track 413. The electronic device defaults that the added picture-in-picture track 413 has a duration of 3s. Optionally, the electronic device can also change the duration of the picture-in-picture track 413. Among them, the picture-in-picture track 413 includes left and right handles, indicating that the picture-in-picture track 413 is selected.

[0083] The electronic device can respond to the click operation on the key frame adding control 414 to add a key frame 421 to the Ath frame of the picture in picture track 413 pointed to by the pointer 415, that is, the picture in picture picture 411B, and display the following: Figure 3C User interface 42 is shown.

[0084] Exemplarily, the picture-in-picture track 422 in the user interface 42 includes a key frame 421 added for the Ath frame. The electronic device can respond to pressing the key frame 421 for the picture-in-picture track 422. Figure 3C Slide in the direction of the arrow shown in the Figure 3D The user interface 43 is shown. In the embodiment of the present application, the above sliding operation on the picture-in-picture track 422 is only an exemplary description, and more other operations may be included to change the picture pointed by the pointer, which is not limited to this.

[0085] For example, the pointer 431 in the user interface 43 points to the Dth frame in the picture-in-picture track 432, and the video screen 433A shows Figure 2G The electronic device can respond to a click operation on the opacity option 434A in the effect option bar 434 to display the following: Figure 3E User interface 44 is shown.

[0086] Exemplarily, the user interface 44 is a user interface for adjusting the opacity of the picture provided by the video editing application. The user interface 44 may include an opacity slider 441. After the electronic device adds a picture-in-picture, the default opacity of the added picture-in-picture is 100%, that is, the added picture-in-picture is completely opaque. Figure 3E As shown, the opacity slider 441 shows 100%, that is, the opacity of the picture-in-picture image 442A is 100%. The electronic device can respond to the opacity slider 441. Figure 3E Slide in the direction of the arrow shown to adjust the opacity of the picture-in-picture image 442A and display the following Figure 3F User interface 45.

[0087] Exemplarily, the user interface 45 may also include an opacity slider 451, a cancel control 455, and a confirmation control 456. The opacity slider 451 in the user interface 45 shows 10%, that is, the opacity of the picture-in-picture image 452A is 10%. The picture-in-picture track 453 in the user interface 45 includes a key frame 454 added for the D-th frame. It is worth noting that the above-mentioned change in the opacity of the picture-in-picture image 442A actually changes the parameters of the picture-in-picture image of the D-th frame, and the operation of changing the opacity is equivalent to adding a key frame to the picture-in-picture image of the D-th frame. Figure 3F As shown, the opacity of the PIP image 452A is reduced by 90% relative to the PIP image 442A.

[0088] From the above editing process, it can be seen that the electronic device changes the opacity of the picture-in-picture track and adds two key frames to the picture-in-picture track. Therefore, in addition to changing the opacity of the D-th frame in the picture-in-picture track, the opacity of the pictures between the A-th frame and the D-th frame in the picture-in-picture track is also changed.

[0089] Figure 3G-3J The user interface of an electronic device playing a video after the opacity of a picture in a picture-in-picture track is changed is exemplarily shown.

[0090] Electronic devices can respond to Figure 3F The click operation of the confirmation control 456 is completed to change the opacity of the picture-in-picture track, and the following is displayed: Figure 3G The user interface 46 is shown. The pointer 461 in the user interface 46 points to the Ath frame in the video track 462 and the picture-in-picture track 463. Among them, no track is selected in the user interface 46. The electronic device can respond to the click operation on the play control 464 and play the video including the video track 462 and the picture-in-picture track 463 in the video preview area 465.

[0091] When the video plays to the Cth frame, the electronic device can display the following Figure 3H The user interface 47 is shown. The pointer 471 in the user interface 47 points to the C-th frame in the video track 472 and the picture-in-picture track 473, and the video screen 474A shows the C-th frame in the video track 472, and the picture-in-picture screen 474B shows the C-th frame in the picture-in-picture track 473. The opacity of the picture-in-picture screen 474B is reduced by 30% relative to the C-th frame of the picture-in-picture track whose opacity is not changed, that is, the opacity of the picture-in-picture screen 474B is 70%.

[0092] When the video plays to the Eth frame, the electronic device can display the following Fig. 3I The user interface 48 is shown. The pointer 481 in the user interface 48 points to the E-th frame in the video track 482 and the picture-in-picture track 483, and the video screen 484A displays the E-th frame in the video track 482, and the picture-in-picture screen 484B displays the E-th frame in the picture-in-picture track 483. The opacity of the picture-in-picture screen 484B is reduced by 60% relative to the E-th frame of the picture-in-picture track whose opacity is not changed, that is, the opacity of the picture-in-picture screen 484B is 40%.

[0093] When the video plays to the Dth frame, the electronic device can display the following Figure 3JThe user interface 49 shown. The pointer 491 in the user interface 49 points to the D-th frame in the video track 492 and the picture-in-picture track 493, and the video screen 494A displays the D-th frame in the video track 492, and the picture-in-picture screen 494B displays the D-th frame in the picture-in-picture track 493. The opacity of the picture-in-picture screen 494B is reduced by 90% relative to the E-th frame of the picture-in-picture track whose opacity is not changed, that is, the opacity of the picture-in-picture screen 494B is 10%.

[0094] When the video plays to the picture after the Dth frame, since the picture-in-picture track is 3 seconds long and there is no picture-in-picture after the Dth frame, the picture displayed in the video preview area is the picture of the video track, and is rotated c degrees counterclockwise relative to the picture in the unrotated video.

[0095] From the above process of playing the video after changing the opacity of the picture-in-picture track, it can be known that the editing of different tracks is independent, and the editing of the added track can be based on the editing of the video track. The electronic device performs corresponding processing on the picture-in-picture track between two key frames, and generates an animation in which the picture of the picture-in-picture track gradually becomes transparent. And in the process of playing the video after the editing process, the picture-in-picture can be gradually transparent on the basis of a relatively smooth and gradual rotation of the video, instead of the picture of the picture-in-picture track suddenly becoming transparent when the video plays to the second key frame.

[0096] Scenario 3: Add a text track based on the video track, edit the text track, and play the edited video.

[0097] The above-mentioned playing of the video after editing may refer to playing the video including the text track after editing and the video track after rotation.

[0098] Figure 4A-4F The user interface of an electronic device adding a text track and changing the position of the picture-in-picture track is exemplarily shown.

[0099] Electronic devices can respond to Figure 2D In the video track 331 Figure 2B Slide in the opposite direction of the arrow shown in the figure to display Figure 4A User interface 50 shown. The electronic device can respond to a click operation on text option 501A in track option bar 501 to enter a user interface for selecting an added text track.

[0100] Exemplarily, the user interface 51 is a user interface for adding a text track provided by a video editing application. The user interface 51 may include, but is not limited to: a text input box 511, a text effect option bar 512, a confirmation control 513, and various text template options 514. The electronic device may display the user interface 51 in response to a click operation on the template 3 option 514A and an operation of inputting the text "hello" in the text input box 511.

[0101] For example, Figure 4B The video preview area 515 of the user interface 51 shown includes two parts: a video screen 515A and a text screen 515B. Video screen 515A may refer to a frame of the video after the above rotation. Text screen 515B is the content of the text "Hello" presented in template 3. Among them, the electronic device defaults that the added text screen is displayed in the center and overlaid on the video screen. Optionally, the electronic device can also be set to overlay the text screen on the video screen, which is not limited. The user interface 51 may include: a video track 516 and a text track 517. The electronic device defaults that the added text track 517 has a duration of 3s. Optionally, the electronic device can also change the duration of the text track 517. Among them, the text track 517 includes left and right handles, indicating that the text track 517 is selected. The electronic device can respond to a click operation on the determined control to complete the addition of the text track and display as shown. Figure 4C User interface 52 is shown.

[0102] The electronic device can respond to the click operation on the key frame adding control 521, add a key frame 531 to the A-th frame picture of the text track 523 pointed to by the pointer 522, that is, the text picture 524A, and display the following: Figure 4D User interface 53 is shown.

[0103] Exemplarily, the text track 532 in the user interface 53 includes a key frame 531 added to the A-th frame. The electronic device can respond to pressing the key frame 531 on the text track 532. Figure 4D Slide in the direction of the arrow shown in the Figure 4E The user interface 54 is shown. In the embodiment of the present application, the above sliding operation on the text track 532 is only an exemplary description, and more other operations may be included to change the screen pointed by the pointer, which is not limited to this.

[0104] For example, the pointer 541 in the user interface 54 points to the Dth frame of the video track 542 and the Dth frame of the text track 543, and the video screen 544A shows Figure 2G The electronic device can respond to pressing the D frame of the video track 362 for the text screen 544B. Figure 4EThe position of the text screen 544B in the video preview area 544 is changed by sliding in the direction of the arrow shown in FIG. Figure 4F The user interface 55 is shown. In the embodiment of the present application, the above-mentioned operation of sliding the text screen 544B is only for exemplary description, and more other operations may be included to change the position of the text screen 544B, which is not limited to this.

[0105] Exemplarily, the text track 551 in the user interface 55 includes a key frame 552 added for the D-th frame. It is worth noting that the above-mentioned change of the position of the text frame 544B in the video preview area 544 actually changes the parameters of the D-th frame text picture, and the operation of changing the position is equivalent to adding a key frame for the D-th frame text picture. Figure 4F As shown, the text screen 553A moves to the left by a distance X1 relative to the text screen 544B.

[0106] From the above editing process, it can be seen that the electronic device changes the position of the text track and adds two key frames in the text track. Therefore, in addition to changing the position of the Dth frame in the text track, the positions of the frames between the Ath frame and the Dth frame in the text track are also changed.

[0107] Figure 4G-4J The user interface of the video after the electronic device changes the position of the picture in the text track is exemplarily shown.

[0108] Electronic devices can respond to Figure 4F Chinese track 551 press Figure 4D Slide in the direction of the arrow to display the following Figure 4G The user interface 56 is shown. The pointer 561 in the user interface 56 points to the Ath frame in the video track 562 and the text track 563. In the user interface 56, no track is selected. The electronic device can respond to a click operation on the play control 564 and play the video including the video track 562 and the text track 563 in the video preview area 565.

[0109] When the video plays to the Cth frame, the electronic device can display the following Figure 4H The user interface 57 shown in FIG. The pointer 571 in the user interface 57 points to the C-th frame in the video track 572 and the text track 573, and the video screen 574A shows the C-th frame in the video track 572, and the text screen 574B shows the C-th frame in the text track 573. The text screen 574B moves to the left by a distance of X1 / 3 relative to the C-th frame of the text track whose position has not changed.

[0110] When the video plays to the Eth frame, the electronic device can display the following Fig. 4I The user interface 58 shown in FIG. 5 shows a pointer 581 in the user interface 58 pointing to the E-th frame in the video track 582 and the text track 583, and the video screen 584A shows the E-th frame in the video track 582, and the text screen 584B shows the E-th frame in the text track 583. The text screen 584B moves 2X1 / 3 to the left relative to the E-th frame of the text track whose position has not changed.

[0111] When the video plays to the Dth frame, the electronic device can display the following Figure 4J The user interface 59 shown in FIG. 5 shows a pointer 591 in the user interface 59 pointing to the D-th frame in the video track 592 and the text track 593, and the video screen 594A shows the D-th frame in the video track 592, and the text screen 594B shows the D-th frame in the text track 593. The text screen 594B moves to the left by a distance X1 relative to the D-th frame of the text track whose position has not changed.

[0112] When the video plays to the picture after the Dth frame, since the text track is 3s long and there is no text after the Dth frame, the picture displayed in the video preview area is the picture of the video track, and it is rotated c degrees counterclockwise relative to the picture in the unrotated video.

[0113] From the above process of playing the video after changing the position of the text track, it can be known that the editing of different tracks is independent, and the editing of the added track can be based on the editing of the video track. The electronic device performs corresponding processing on the text track between two key frames, and generates an animation in which the picture of the text track gradually becomes transparent. And in the process of playing the video after the editing process, the text can gradually move its position on the basis of the relatively smooth and gradual rotation of the video, instead of the picture of the text track suddenly moving a certain position when the video is played to the second key frame.

[0114] Based on the process of editing videos and playing the edited videos in various scenarios introduced above in combination with the user interface, a flow chart of the video editing method provided in an embodiment of the present application is introduced below.

[0115] Figure 5 The following is an exemplary diagram of a video editing method. Figure 5 The method shown is a method for editing based on a single track, that is, a method for editing one track, namely, a video track.

[0116] like Figure 5 As shown, the method includes:

[0117] S501: Display a first user interface, where the first user interface includes a first track, and multiple frames of a first video are displayed in the first track.

[0118] The above-mentioned first user interface may refer to a user interface for editing videos in an electronic device. For example, the first user interface may be a user interface provided by a video editing application in an electronic device. In an embodiment of the present application, the application to which the first user interface belongs may be a system application or a third-party application, without limitation. And the name of the application to which the first user interface belongs is not limited. Multiple frames of the first video are displayed in the above-mentioned first track. The first video may refer to a video stored in an electronic device or a video shot by an electronic device.

[0119] In some implementations, the electronic device may display the first user interface in response to a user operation. The user operation may include: a click operation, a slide operation, a voice command, a gesture command, etc. Exemplarily, the electronic device may enter the first user interface in response to a click operation on an edit control for a first video in a gallery application.

[0120] In some implementations, the first user interface may further include a video preview area. Specifically, the electronic device may play the first video in the video preview area, and may also edit the first video in the video preview area. Playing the first video may refer to continuously displaying each frame of the first video; editing the first video may refer to editing one or more frames of the first video. Exemplarily, the electronic device may play the first video in the video preview area in the first user interface, and may also edit a frame of the first video displayed in the video preview area.

[0121] In some implementations, the electronic device can change a frame of the first video displayed in the video preview area in response to a user operation. Exemplarily, the first user interface may include a fixed pointer in addition to the video preview area. The length of the first track can be used to indicate the duration of the first video, and the position of the first track indicated by the pointer indicates a time point (i.e., moment) in the first video, and the video preview area displays a frame of the first video at the above moment. The electronic device can change the position of the first track indicated by the pointer in response to a sliding operation on the first track. That is, a frame of the first video displayed in the video preview area can be changed.

[0122] Exemplarily, the first user interface may refer to Figure 2A The user interface 30 shown. The video preview area in the first user interface may refer to Figure 2AThe video preview area 304 is shown. The first track may refer to the video track 303, and the fixed pointer may refer to the pointer 302. The first video may refer to Figure 1B The video corresponding to the video control 214 is shown. In the embodiment of the present application, the content included in the above-mentioned first user interface is only for exemplary description, and may also include more or less content.

[0123] S502: Detecting an operation of adding a key frame at a first moment of a first video, obtaining first parameters of the picture at the first moment, which may include but are not limited to: display position, size, rotation angle, opacity and other parameters of the picture at the first moment.

[0124] In some implementations, before the electronic device detects an operation to add a keyframe for the first moment of the first video, the first user interface may display a picture of the first moment in the first video. Exemplarily, the electronic device may perform a sliding operation on the first track to change the position in the first track pointed to by the pointer, so that the pointer points to the position of the first moment of the first track. The electronic device may then display the picture of the first moment in the first video in the video preview area. And, at this time, the electronic device has selected the video track. In the embodiment of the present application, the operation of causing the first user interface to display the picture of the first moment in the first video is not limited.

[0125] In some implementations, the electronic device may add a key frame to the first moment of the first video only after the first track is selected.

[0126] The first moment mentioned above may refer to any frame in the first video. However, since a frame after the first moment will be edited later, the first moment here generally does not refer to the last frame of the first video.

[0127] Adding a key frame to the first moment of the first video may refer to adding a mark to the first moment of the first video. After the electronic device detects the operation of adding a key frame, it can not only display the mark of the first key frame at the first moment of the first track, but also store the timestamp information of the first key frame. For example, when the first moment is 2.5s, the timestamp information of the first key frame is 2500ms.

[0128] The function of a key frame may mean that the electronic device can perform segmented editing processing on a video. Generally speaking, when a video includes only one key frame, the effects of all the pictures in the video including only one key frame are consistent, and a key frame animation cannot be generated. In other words, when a video includes only one key frame, the key frame is invalid. The purpose of two or more key frames is not introduced here, and reference may be made to the relevant description in S503 below.

[0129] In some implementations, the electronic device may obtain the specific content displayed on the screen at the first moment, and may also obtain a first parameter of the screen at the first moment. The first parameter may refer to a parameter used to describe the screen at the first moment, such as a position, size, rotation angle, opacity, and other parameters of the screen at the first moment.

[0130] In some implementations, the first parameter may be used to indicate the effect of the picture at the first moment.

[0131] The following specifically introduces the expression form of the parameters used to describe the picture. The electronic device can set the type of various parameters to float floating point type for storage.

[0132] Exemplarily, the position parameter can be represented by the coordinates moved along the x and y axes relative to the position of the default screen, the sign of the coordinates moved in the negative direction of the x or y axis is "-", and the sign of the coordinates moved in the positive direction of the x or y axis is "+", and the unit of the coordinates is px. The size parameter can be represented by the scale of reduction or enlargement relative to the size of the default screen. The rotation angle parameter can be represented by the degree of counterclockwise or clockwise rotation relative to the rotation angle of the default screen, the sign of the degree of counterclockwise rotation is "-", and the sign of the degree of clockwise rotation is "+", and the unit of degree is °. The opacity parameter can be represented by a numerical value that increases or decreases the opacity relative to the opacity of the default screen. The range of the numerical value used to represent the opacity parameter is between 0 and 100.

[0133] In the embodiment of the present application, the parameters used to describe the picture are not limited to the above types, and may also include more types of parameters, which are not limited to this.

[0134] Exemplarily, when the picture at the first moment displayed by the electronic device is default, that is, the electronic device has not edited the picture at the first moment, the display position of the picture at the first moment can be expressed as (0px, 0px), the size of the picture at the first moment can be expressed as 1, the rotation degree of the picture at the first moment can be expressed as 0°, and the opacity of the picture at the first moment can be expressed as 100.

[0135] In some implementations, parameters of a picture may be used to describe the effect of the picture.

[0136] In a possible implementation, the electronic device may edit the picture at the first moment before detecting the operation of adding a key frame at the first moment of the first video. The electronic device may obtain various parameters for describing the picture at the first moment after the editing process. In this case, the various parameters for describing the picture at the first moment after the editing process may be referred to as first parameters. The editing process may include, but is not limited to: rotating the picture, scaling the picture, changing the display position of the picture, changing the opacity of the picture, and the like. For the editing process, reference may be made to the relevant description in S503, which will not be described here.

[0137] In another possible implementation, after detecting the operation of adding a key frame at the first moment of the first video, the electronic device may edit the picture at the first moment. Then the electronic device obtains various parameters for describing the picture at the first moment after the editing process. Similarly, in this case, the various parameters for describing the picture at the first moment after the editing process may be referred to as first parameters.

[0138] For example, the above operation of adding a key frame may refer to acting on Figure 2A In the key frame adding control 301, the first moment of the first video may refer to Figure 2A The picture displayed in the video preview area 304.

[0139] S503, detecting an operation of editing the picture at the second moment of the first video, and obtaining second parameters of the picture at the second moment after editing, wherein the second parameters may include but are not limited to: display position, size, rotation angle, opacity and other parameters of the picture at the second moment after editing.

[0140] In some implementations, before the electronic device detects an operation to edit the screen at the second moment of the first video, the first user interface may switch from displaying the screen at the first moment in the first video to displaying the screen at the second moment in the first video. Exemplarily, the electronic device may perform a sliding operation on the first track to change the position of the pointer in the first track, so that the pointer switches from pointing to the first moment of the first track to pointing to the second moment of the first track. Then, the electronic device may display the screen at the second moment in the first video in the video preview area. And, at this time, the electronic device has selected the first track. In an embodiment of the present application, the operation of causing the screen at the second moment in the first video to be displayed in the first user interface is not limited.

[0141] The second moment may refer to any frame in the first video that is different from the first moment. Furthermore, the second moment may refer to the last frame in the first video. Specifically, the second moment may be before the first moment or after the first moment. Figure 5 In the flowchart shown, the video editing method is introduced by taking the second moment being located after the first moment as an example.

[0142] The above-mentioned operation of editing the picture at the second moment of the first video may include two situations:

[0143] Case 1: The electronic device directly edits the image at the second moment, and then the electronic device can display the second key frame at the second moment in the first track. The editing process includes one or more of the following: rotating the image, scaling the image, changing the display position of the image, changing the opacity of the image, etc.

[0144] Case 2: In response to the operation of adding a key frame at the second moment of the first video, after the electronic device displays the second key frame at the second moment in the first track, the electronic device edits the picture at the second moment.

[0145] The following describes in detail how to edit the screen.

[0146] Exemplarily, when the electronic device displays the picture at the second moment in the video preview area in the first user interface, the electronic device can rotate the picture at the second moment. For example, the electronic device can respond to the operation of long pressing with two fingers on the picture at the second moment in the video preview area and rotating the two fingers counterclockwise or clockwise at the same time, and rotate the picture at the second moment counterclockwise or clockwise around the midpoint of the picture at the second moment by a certain number of degrees. Exemplarily, the operation of rotating the picture can refer to Figure 2C The operation shown.

[0147] The electronic device may also move the display position of the second moment picture. For example, in response to a long press and a sliding operation in any direction on the second moment picture displayed in the video preview area, the electronic device may move the second moment picture in the sliding direction by a certain position. Exemplarily, the operation of changing the picture position may refer to the following example. Figure 4E The operation shown.

[0148] The electronic device can also zoom in or out the screen at the second moment. For example, the electronic device can zoom in or out the second screen in response to a two-finger press on the screen at the second moment displayed in the video preview area and a sliding operation in which one finger is in one direction and the other finger is in the opposite direction of the one direction. Exemplarily, the operation of zooming in or out the screen at the second moment can refer to the operation shown in 2I.

[0149] The electronic device may also change the opacity of the image at the second moment.

[0150] In the embodiment of the present application, the editing of the screen is not limited to the above-mentioned types, but may also include more types of editing processes, and the operations for implementing the above-mentioned types of editing processes are not limited.

[0151] In the embodiment of the present application, the electronic device can perform one of the above-mentioned types of editing processing on the picture at the second moment, or can perform multiple types of editing processing. Moreover, the effects of multiple types of editing can be superimposed.

[0152] In some implementations, after detecting an editing operation for the second moment of the first video, similarly, the electronic device can not only display a mark of the second key frame at the second moment of the first track, but also store the timestamp information of the second key frame. For example, when the second moment is 7.5 seconds, the timestamp information of the second key frame is 7500ms.

[0153] In addition, similarly, the electronic device can also obtain the specific content displayed on the screen at the second moment before editing and the second parameter of the screen at the second moment after editing. The second parameter may refer to a parameter used to describe the screen at the second moment after editing, such as the position, size, rotation angle, and opacity of the screen at the second moment after editing. In some implementations, the second parameter may be used to indicate the effect of the screen at the second moment of the first video after editing.

[0154] Exemplarily, the expression form of the second parameter may refer to the relevant description in the above S502, which will not be described in detail here.

[0155] For example, the above-mentioned editing operation may refer to acting on Figure 2C as well as Fig.2I In the editing operation, the second moment of the first video may refer to Figure 2C The picture displayed in the video preview area 323 in the second moment after editing may refer to Figure 2D The picture displayed in the video preview area 333.

[0156] S504: Obtain parameters of each frame in the second video according to the first parameter and the second parameter, and generate the second video based on the obtained parameters of each frame, wherein the second video is the first video after editing.

[0157] In some implementations, the pictures included in the second video can be divided into three categories: multiple pictures before the first moment, multiple pictures between the first moment and the second moment, and multiple pictures after the second moment. Among them, the parameters of the picture at the first moment in the second video can refer to the first parameter, and the parameters of the picture at the second moment in the second video can refer to the second parameter. In addition, the parameters of the multiple pictures before the first moment are consistent with the first parameter. The parameters of the multiple pictures after the second moment are consistent with the second parameter. It can be seen that the electronic device also needs to obtain the parameters of the multiple pictures between the first moment and the second moment to generate the second video.

[0158] In some implementations, after acquiring the second parameter of the picture at the second moment after editing, the electronic device may acquire the parameters of multiple pictures between the first moment and the second moment after editing. Since the first parameter includes the parameter for describing the picture under the first key frame, and the second parameter includes the parameter for describing the picture under the second key frame, the acquisition of the parameters of multiple pictures according to the first parameter and the second parameter can be understood as acquiring the parameters of multiple pictures according to the time information of the first key frame, the time information of the second key frame, the first parameter, and the second parameter.

[0159] Combine the following Figure 6 The following describes a calculation rule for obtaining parameters of multiple images between a first moment and a second moment.

[0160] For example, Figure 6 As shown, the duration of the second video is 10s, the first moment M1 (i.e., the first key frame) is located at the 1s, and the second moment M2 (i.e., the second key frame) is located at the 9s. It can be seen from the above that the parameters of the multiple frames between the 0s and 1s in the second video are all first parameters. The parameters of the multiple frames between the 9s and 10s in the second video are all second parameters. Exemplarily, it is assumed that the position parameter in the first parameter is (0px, 0px), the size parameter is 1, the rotation angle parameter is 0°, and the opacity parameter is 100; the position parameter in the second parameter is (120px, 160px), the size parameter is 2.2, the rotation angle parameter is +160°, and the opacity parameter is 20.

[0161] In some implementations, the parameters of the multiple images between the first moment and the second moment are actually parameter information of point Y. Exemplarily, the electronic device can obtain the parameter information of Y according to the following formula.

[0162] P1=(MY-M1) / (M2-M1);

[0163] CY=(C2-C1)*P1+C1;

[0164] Among them, P1 represents the proportion of point Y between the first moment and the second moment, MY represents the moment at which point Y is located, M1 represents the first moment, M2 represents the second moment, CY represents the parameter of point Y, C1 represents the parameter of the first moment, and C2 represents the parameter of the second moment.

[0165] For example, Figure 6 As shown, point Y is at the 3rd second. Based on the above two formulas, we can get:

[0166] Proportion P1 = (3-1) / (9-1) = 0.25;

[0167] The position parameter of point Y CY1 = ((120px, 160px) - (0px, 0px)) * 0.25 + (0px, 0px) = (30px, 40px);

[0168] The size parameter of point Y CY2 = (2.2-1)*0.25+1 = 1.3;

[0169] The rotation parameter of point Y CY3 = (+160°-0°)*0.25+0°=40°;

[0170] The rotation parameter of point Y CY4=(100-20)*0.25+20=40.

[0171] In some implementations, the electronic device can obtain the parameters of multiple frames between the first moment and the second moment in the second video using the above two formulas. Since the electronic device stores the specific contents displayed by each frame that has not been edited, the electronic device can generate the second video based on the above-obtained parameters of each frame in the second video and the contents displayed by each frame.

[0172] In a possible implementation, the electronic device may generate the second video when the edited second screen is displayed in the first user interface. The electronic device may generate the second video after the edited second screen is displayed in the first user interface. For example, in S505, before playing each frame of the second video, the electronic device may calculate the parameters of the frame to be played in real time according to the above two formulas, and generate the frame to be played according to the specific content and parameters of the frame to be played, and then render the frame to be displayed on the screen.

[0173] Optionally, the electronic device may store the generated second video in the electronic device. Figure 2D The export control shown stores the edited video in the electronic device. Optionally, the electronic device can also replace the original first video with the generated second video.

[0174] In a possible implementation, a video may include not only two key frames, but more than two key frames. In this case, the electronic device can also use the above two formulas to calculate the parameters of the picture between every two key frames, which will not be described in detail.

[0175] In a possible implementation, the above-mentioned calculation rule for obtaining the parameters of multiple pictures between the first moment and the second moment allows the multiple pictures to change evenly and smoothly. It can also be implemented based on other formulas. For example, there is a calculation rule that allows the multiple pictures to change unevenly and smoothly.

[0176] S505: Play the second video.

[0177] In some implementations, after the electronic device generates the second video, the first track included in the first user interface can display multiple frames of the second video, and the electronic device can play the second video in the first user interface. Specifically, the electronic device can start playing from the beginning of the second video, or from a certain frame in the second video. Exemplarily, the electronic device can respond to a sliding operation on the first track to change the position of the first track pointed to by the pointer, that is, to change the position where the second video starts to be played. Also exemplarily, the electronic device can play the second video in the first user interface by sliding the first track, and can also play the second video by responding to a click operation on the playback control. Exemplarily, the electronic device can respond to a sliding operation on the first track to change the position of the first track pointed to by the pointer, that is, to change the position where the second video starts to be played. Also exemplarily, the electronic device can play the second video in the first user interface by sliding the first track, and can also play the second video by responding to a click operation on the playback control. Figure 2E Click the play control 343 to play the second video.

[0178] In other implementations, the electronic device may also play the second video stored in the electronic device in the gallery. Similarly, the electronic device may start playing from the beginning of the second video, or may start playing from a certain frame in the second video. Exemplarily, the electronic device may respond to a click operation on the control for the second video in the gallery, enter a user interface that displays the second video in full screen, and then respond to a click operation on the play control to play the second video.

[0179] For example, the user interface of the electronic device playing the second video can refer to Figure 2E-2H The user interface shown.

[0180] By the above Figure 5As can be seen from the process method shown, the electronic device can add key frames to the video, edit one or more frames in the video, and generate parameters of multiple frames between any two key frames in the video according to the above two formulas. In this way, when the electronic device plays the edited video, it can be observed that the video is relatively smooth, and gradually rotates, scales, moves, changes in opacity, etc., so that the user has a good viewing experience.

[0181] In a possible implementation, the video editing method provided by the embodiment of the present application is not limited to the above method of adding two key frames to the video, and multiple key frames can also be added to the video, wherein the electronic device generates a key frame animation between every two adjacent key frames, that is, the effect of the picture between every two adjacent key frame pictures can refer to the above method. Figure 5 The method shown will not be described in detail here.

[0182] Based on the flowchart shown above, the following is a flowchart of the video editing method provided in an embodiment of the present application.

[0183] Figure 7 The following is an exemplary flow chart of another video editing method. Figure 7 The method shown is a method for editing a video based on the video including multiple material tracks. In the embodiment of the present application, the editing process for different tracks may be different, and the editing process for a certain track will not affect another track.

[0184] like Figure 7 As shown, the method includes:

[0185] S701: Display a first user interface, where the first user interface includes a first track, and the first track displays multiple frames of a first video.

[0186] The first user interface may refer to a user interface in an electronic device for editing a video. For example, the first user interface may be a user interface provided by a video editing application in the electronic device. In an embodiment of the present application, the application to which the first user interface belongs may be a system application or a third-party application, without limitation. The name of the application to which the first user interface belongs is not limited. The first video may refer to a video stored in an electronic device or a video shot by the electronic device.

[0187] In some implementations, the electronic device may display the first user interface in response to a user operation. The user operation may include: a click operation, a slide operation, a voice command, a gesture command, etc. Exemplarily, the electronic device may enter the first user interface in response to a click operation on an editing control for a video in a gallery application. Similarly, the first user interface may also include a video preview area and a fixed pointer.

[0188] In some implementations, since different material tracks will be added later, in order to distinguish the material tracks, the video track is referred to as the first track. The video track here may refer to the unprocessed main video material stored in the electronic device, such as Figure 5 The first video in Figure 6 Take the first track displaying multiple frames of the first video as an example.

[0189] In another implementation, the first track may also be used to indicate video material after editing, for example Figure 5 In the embodiment of the present application, the editing process for a certain track will not affect the subsequent editing process for another track. Therefore, whether the video displayed in the first track has been edited will not affect the subsequent editing process for the second track.

[0190] Exemplarily, the first user interface may refer to Figure 3A The user interface 40 shown. The first track may refer to Figure 3B The video track 412 is shown in .

[0191] S702: In response to the user operation, a second track is added, wherein multiple frames of the first material are displayed in the second track. The second track may include but is not limited to: a picture-in-picture, text, sticker, and other tracks.

[0192] In some implementations, after the first track is displayed in the electronic device, a second track may be added. The electronic device may add one or more second tracks, wherein the second track displays multiple frames of the first material.

[0193] In the embodiment of the present application, the electronic device does not limit the relative position of the second track and the first track. The second track can be located above the first track or below the first track. That is, the second track can partially or completely cover the first track, or be partially or completely covered by the first track. The electronic device does not limit the duration of the second track. The duration of the second track can be greater than the duration of the first track, or it can be less than the duration of the first track. The electronic device does not limit the starting position of the second track. The starting position of the second track can be located before the starting position of the first track, or it can be located after the starting position of the first track. It is worth noting that the various parameters of the picture of the second track added by the electronic device (such as position, size, rotation angle, and opacity) are the same as Figure 5 The default parameters are the same as shown in the first video.

[0194] Exemplarily, the electronic device defaults that the added second track covers the first track, the default duration of the added second track is 3s, and the default starting position of the added second track is consistent with the starting position of the first track. The electronic device can change the above default situations, for example, it can respond to user operations to make the first track cover the second track, and it can also respond to user operations to change the duration of the second track, and it can also respond to user operations to change the starting position of the second track.

[0195] In some implementations, the electronic device can add one or more second tracks. Figure 6 In the method flow shown, adding a second track is used as an example. Since different tracks do not affect each other, the method flow based on multiple second tracks can refer to Figure 7 The method flow is shown.

[0196] The above-mentioned second track may refer to various types of material tracks other than the video track. Specifically, various types of materials may include but are not limited to: picture-in-picture, text, stickers, etc. Among them, the picture-in-picture material may refer to multiple frames of static images, or multiple frames of dynamic images. And the picture-in-picture material can form a picture-in-picture effect with the image of the main video. Text material may refer to a picture whose displayed content is a text content containing a template, or it may refer to a static picture or a dynamic picture. Sticker material may refer to a picture used for decoration to make the display richer, or it may refer to a static picture or a dynamic picture. Exemplarily, the picture of the picture-in-picture track can refer to the above Figure 3B The picture-in-picture screen 411B shown. The picture of the text track can refer to the above Figure 4B Text screen 515B is shown.

[0197] In some implementations, adding a text track also requires the user to input text information, for example, the text "Hello" information input by the user.

[0198] In the embodiment of the present application, the above-mentioned tracks are only illustrative, and more types of material tracks may also be included.

[0199] Exemplarily, the user operation for adding the second track may refer to the operation on Figure 3A The click operation of the picture-in-picture option 401A in the picture-in-picture option 401A may also refer to the click operation on Figure 4A Click operation of Chinese character option 501A.

[0200] The second track may refer to the picture-in-picture track 413 in 3B, or may refer to Figure 4B Text track 417.

[0201] S703. Detect an operation of adding a keyframe for the third moment of the first material, and obtain third parameters of the picture of the first material at the third moment. The third parameters may include but are not limited to: display position, size, rotation angle, opacity and other parameters of the picture of the first material at the third moment.

[0202] In some implementations, when the electronic device detects an operation of adding a key frame for the third moment of the second track, the first user interface may display a picture of the third moment in the second track. Exemplarily, the electronic device may act on a sliding operation of the second track to change the position in the second track pointed to by the pointer so that the pointer points to the third moment of the second track. The electronic device may then display the picture of the third moment in the second track in the video preview area. And, at this time, the electronic device has selected the second track. It is worth noting that a key frame can be added to a certain moment of the second track only after the second track is selected. In the embodiment of the present application, the operation of causing the first user interface to display a picture of the third moment in the second track is not limited.

[0203] The third moment mentioned above may refer to any frame in the second track. However, since a frame after the third moment will be edited later, the third moment here generally does not refer to the last frame of the second track.

[0204] Adding a key frame for the third moment may refer to adding a mark to the third moment of the second track. After the electronic device detects the operation of adding a key frame, it can not only display the first key frame mark at the position of the third moment of the second track, but also store the timestamp information of the first key frame. For example, when the third moment is 2.5s, the timestamp information of the first key frame is 2500ms.

[0205] In some implementations, the electronic device can obtain the specific content displayed on the screen at the third moment of the second track, and can also obtain the third parameter of the screen at the third moment of the second track. The third parameter can refer to the parameters used to describe the screen at the third moment of the second track, such as the display position, size, rotation angle, and opacity of the screen at the third moment of the second track. For a detailed introduction to the various parameters used to describe the screen, please refer to the above Figure 5 The relevant description in S502 is not repeated here.

[0206] In some implementations, the third parameter may be used to indicate the effect of the picture at the third moment of the first material, and the effect of the picture at the third moment of the second track may be referred to as the first effect.

[0207] Similarly, in a possible implementation, the electronic device may edit the image at the third moment before detecting that a key frame is added at the third moment of the second track.

[0208] Similarly, in another possible implementation, the electronic device can edit the picture at the third moment after detecting that a key frame is added at the third moment of the second track. The various parameters used to describe the picture at the third moment after the editing process can be referred to as third parameters.

[0209] For example, the above operation of adding a key frame may refer to acting on Figure 3B The click operation of adding the key frame control 414 in the middle, the picture of the third moment of the first material can refer to Figure 3B The picture-in-picture image 411B displayed in the video preview area 411 .

[0210] S704. Detect an editing operation on the picture at the fourth moment of the first material, and obtain fourth parameters of the picture at the fourth moment of the first material after editing. The fourth parameters may include: display position, size, rotation angle, opacity and other parameters of the picture at the fourth moment of the first material after editing.

[0211] This step S704 is similar to S503. For a detailed introduction to S704, please refer to the relevant description in the above S503, which will not be repeated here.

[0212] For example, the above-mentioned editing operation may refer to acting on Figure 3E The opacity slider 441 in the slide operation may refer to the fourth moment of the first material. Figure 3E The picture-in-picture picture 442B, the picture of the fourth moment of the first material after editing, may refer to Figure 3F Picture-in-picture image 452A.

[0213] S705: Obtain parameters of each frame in the second material according to the third parameter and the fourth parameter, and generate the second material based on the obtained parameters of each frame, wherein the second material is the first material after editing.

[0214] This step S705 is similar to S504. For a detailed introduction to S705, please refer to the relevant description in the above S504, which will not be repeated here.

[0215] S706: In response to the operation of playing the video, play the video including the first video and the second material.

[0216] In some implementations, after the electronic device generates the second material, the electronic device can respond to the user operation and play the video containing the first video and the second material. The electronic device can play the video in the first user interface or in the gallery. For a specific description of playing the video, reference can be made to the relevant description in S505 above, which will not be repeated here.

[0217] It is worth noting that, since the starting position of the second material may or may not be consistent with the starting position of the first video, the playing of the video is different from that in the above S505.

[0218] Exemplarily, the first video has a duration of 10 seconds, the second material has a duration of 3 seconds, and the starting position of the second material is before the starting position of the first video, for example, the starting position of the second material is 2 seconds before the starting position of the first video, and the total duration of the video played by the electronic device is 12 seconds. The electronic device plays the 0th to 2nd images of the second material between the 0th and 2nd seconds after the start, plays the 2nd to 3rd images of the second material and the superimposed images of the 0th to 1st images of the first track between the 2nd and 3rd seconds, and plays the images of the 1st to 10th images of the first video between the 3rd and 12th seconds.

[0219] In a possible implementation, after the electronic device generates the second material, the electronic device may replace the template in the second material, and the second material after the template is replaced still includes the third key frame and the fourth key frame. Figure 4B After the template 3514A is generated, Figure 4G-4J After the second material is displayed, the electronic device can select text option 501A again to replace the module of the second material with Figure 4B In template 4, the picture content of the second material after replacement changes with the change of the template, but the picture effect of the second material after replacement is consistent with the picture effect of the second material before replacement.

[0220] By the above Figure 6 It can be seen from the process method shown that the electronic device can add various material tracks, add key frames to various material tracks, edit one or more frames in the material track, and generate parameters of multiple pictures between any two key frames in the track according to the above two formulas. In this way, when the electronic device plays the edited video, it can be observed that the changes between different tracks are independent, and the material track is relatively smooth, gradually rotated, scaled, displayed, moved, opacity changed, etc., so that the user has a good viewing experience.

[0221] Next, combine Figure 8 as well as Fig. 9 To introduce the previous article Figure 5 as well as Figure 7 An operating system (OS) interaction diagram of an electronic device corresponding to the method flow in FIG.

[0222] Figure 8 as well as Fig. 9 The user operation module, data control module (operation), data storage module, and data rendering module shown are modules included in the electronic device for managing the images in the edited video. For the definitions of these modules, please refer to the detailed description of the electronic device software architecture later. The specific functions of these modules have been recorded in detail in the following process and will not be repeated here.

[0223] like Figure 8 As shown, the OS interaction of the editing method based on a video containing only a single track provided in the embodiment of the present application includes the following steps:

[0224] S801: A user operation module detects an operation of adding a key frame at a first moment.

[0225] In some implementations, the user operation module may acquire the first parameter of the picture at the first moment after detecting the operation of adding a key frame at the first moment of the first video. Exemplarily, the user operation module may detect a click operation for adding a key frame. For the relevant introduction of S801, reference may be made to the detailed description of S502 above.

[0226] Optionally, before S801, the user operation module may also detect an editing operation on the screen at the first moment, and obtain a first parameter of the screen at the first moment after editing.

[0227] S802: The user operation module sends a first parameter of a picture at a first moment to a data storage layer in a data storage module.

[0228] In some implementations, after obtaining the first parameter, the user operation module may send it to the data storage layer in the data storage module via the data control module and the data transmission layer in the data storage module. Exemplarily, the data control module may pass the first parameter to the data storage layer via the setKeyFrameParams() method.

[0229] S803: The data storage layer stores the first parameter.

[0230] In some implementations, the data storage layer stores the first parameter in a material data class (Material) through a data visitor. Exemplarily, other modules in the electronic device can read the first parameter at the first moment from the data storage layer through the identifier of the first video and the timestamp identifier of the key frame. Optionally, the material data class can also store the content of the picture at the first moment.

[0231] When the user operation module detects that the first parameter is inconsistent with the default parameter in the first video, S804 and S805 may be executed.

[0232] Optionally, S804, the data storage layer sends the first parameter to the Animation in the data rendering module.

[0233] In some implementations, the data storage layer may send the first parameter and the content of the screen at the first moment to the Animation via the data rendering layer. Exemplarily, the data rendering layer may send the first parameter to the Animation via setTransformParam().

[0234] Optionally, S805, Animation renders and generates a picture of the first moment of the second video based on the first parameter.

[0235] In some implementations, the Animation may render and generate the screen at the first moment after editing according to the first parameter and the content of the screen at the first moment.

[0236] S806: The user operation module detects an operation of editing the picture at the second moment.

[0237] In some implementations, the user operation module may, after detecting the operation of editing the second moment of the first video, obtain the second parameter of the picture at the second moment after editing. The above-mentioned operation of editing the second moment of the first video is equivalent to the operation of adding a key frame at the second moment. Exemplarily, the user operation module may detect the editing operation of the picture at the second moment. For the relevant introduction of S806, reference may be made to the detailed description of S503 above.

[0238] S807: The user operation module sends the second parameter of the picture at the second moment to the data storage layer.

[0239] In some implementations, after acquiring the second parameter, the user operation module may send it to the data storage layer via the data control module and the data transmission layer.

[0240] S808. The data storage layer stores the second parameter.

[0241] In some implementations, the data storage layer stores the second parameter in the material data class through data monitoring. Exemplarily, other modules in the electronic device can read the second parameter at the second moment from the data storage layer through the identifier of the first video and the timestamp identifier of the key frame. Optionally, the material data class can also store the content of the picture at the second moment.

[0242] S809: The data storage layer sends the second parameter to Animation.

[0243] In some implementations, the data storage layer may send the second parameter and the content of the picture at the second moment to the Animation via the data rendering layer.

[0244] S810, Animation renders and generates a picture of a second video at a second moment based on the second parameters.

[0245] In some implementations, the Animation may render and generate the edited picture at the second moment according to the second parameter and the content of the picture at the second moment.

[0246] S811. The user operation module detects an operation of playing a video.

[0247] Exemplarily, the user operation module may detect a click operation on the play control. For the relevant introduction of S811, reference may be made to the detailed description of S505 above.

[0248] S812: The user operation module sends instruction information for playing the second video to the data rendering layer.

[0249] S813: The data rendering layer determines whether the operation in S806 is valid according to the first parameter and the second parameter.

[0250] The above-mentioned validity actually means that the first parameter is inconsistent with the second parameter. That is, the first video is effectively edited. This is because a video containing only one key frame for a period of time does not have a practical function. In order to generate a multi-frame picture that is automatically edited by the user editing one or two frames, the video needs to contain two or more key frames.

[0251] S814. The data rendering layer sends instruction information for calculating parameters of the Nth frame in the second video to the key frame data processing layer in the data storage module.

[0252] Wherein, N is a non-negative integer, and the Nth frame is any frame in the second video. Since the second video is composed of multiple pictures, it is necessary to render and generate each frame of the second video as the second video is played.

[0253] S815. The key frame data processing layer determines whether the Nth frame is within the first moment and the second moment, and obtains parameters of the Nth frame in the second video according to the first parameter and the second parameter.

[0254] In some implementations, when the Nth frame is before the first moment, the parameters of the Nth frame are the first parameters; when the Nth frame is after the second moment, the parameters of the Nth frame are the second parameters; when the Nth frame is between the first moment and the second moment, the parameters of the Nth frame can be based on the above Figure 6 The two formulas introduced are calculated and will not be repeated here.

[0255] S816. The key frame data processing layer sends information about parameters of the Nth frame in the second video to the data storage layer.

[0256] S817. The data storage layer stores the parameters of the Nth frame in the second video.

[0257] In some implementations, the data storage layer stores the parameters of the Nth frame in the second video in the material data class through data monitoring, so that when the second video is played again later, it can be directly read without executing S815.

[0258] S818. The data storage layer sends the parameters of the Nth frame in the second video to Animation.

[0259] In some implementations, the data storage layer may send the content of the Nth frame in the second video and the frame at the second moment to the Animation via the data rendering layer.

[0260] S819. Animation draws the Nth frame in the second video based on the parameters of the Nth frame in the second video.

[0261] In some implementations, the Animation may render and generate the Nth frame in the edited second video according to the parameters of the Nth frame in the second video and the content of the Nth frame.

[0262] S814-S819 are repeatedly executed until the second video is finished playing.

[0263] like Fig. 9 As shown, the OS interaction of the video editing method based on multiple tracks provided in the embodiment of the present application includes the following steps:

[0264] S901: A user operation module detects an operation of adding a second track, where multiple frames of a first material are displayed.

[0265] S902: The user operation module sends the material identification information of the first material to the data storage layer in the data storage module.

[0266] In some implementations, the data storage layer stores various materials with material identifiers. Figure 4B The middle template 3 514A has a unique material identifier that is different from other templates and other material types.

[0267] It is worth noting that when generating materials still requires user input, the user operation module also needs to send the user's input information to the data storage layer. For example, the text "Hello" input by the user will also be sent to Animation together with the first material default parameters.

[0268] S903: The data storage layer determines a first material default parameter according to the material identification information of the first material.

[0269] S904: The data storage layer sends the first material default parameters to the Animation in the data rendering module.

[0270] S905 , Animation generates a frame of the first material by rendering based on default parameters.

[0271] S906: The user operation module detects an operation of adding a key frame at the third moment of the first material.

[0272] S907: The user operation module sends the third parameter of the picture at the third moment to the data storage layer.

[0273] S908. The data storage layer stores the third parameter.

[0274] Optionally, S909, the data storage layer sends a third parameter to Animation.

[0275] Optionally, S910, Animation renders and generates a picture of the second material at a third moment based on the third parameter.

[0276] S911. The user operation module detects an operation of editing the picture at the fourth moment of the first material.

[0277] S912: The user operation module sends a fourth parameter of the image at a fourth moment to the data storage layer.

[0278] S913. The data storage layer stores the fourth parameter.

[0279] S914. The data storage layer sends the fourth parameter to Animation.

[0280] S915. Animation generates a picture of the second material at a fourth moment based on the fourth parameter rendering.

[0281] S916: The user operation module detects a video playing operation.

[0282] S917: The user operation module sends instruction information for playing the video including the first video and the second material to the data rendering layer.

[0283] S918. The data rendering layer determines whether the operation in S911 is valid according to the third parameter and the fourth parameter.

[0284] S919: The data rendering layer sends instruction information for calculating parameters of the Nth frame in the second material to the key frame data processing layer in the data storage module.

[0285] S920: The key frame data processing layer determines whether the Nth frame is within the third moment and the fourth moment, and obtains parameters of the Nth frame in the second material according to the third parameter and the fourth parameter.

[0286] S921. The key frame data processing layer sends information about parameters of the Nth frame in the second material to the data storage layer.

[0287] S922. The data storage layer stores parameters of the Nth frame in the second material.

[0288] S923: The data storage layer sends the parameters of the Nth frame in the second material to Animation.

[0289] S924. Animation draws the Nth frame in the second material based on the parameters of the Nth frame in the second material.

[0290] Repeat S919-S924 until the video including the first video and the second material is played.

[0291] It is worth noting that the above S919-S924 are used to generate the second material, and the video played by the electronic device includes the first video and the second material. Therefore, when the video played by the electronic device includes the second video and the second material, during the execution of the above S919-S924, S814-S819 are also executed synchronously.

[0292] Based on the above introduction to the process method, the structural diagram of the electronic device provided in the embodiment of the present application is introduced below.

[0293] Fig.10 A schematic structural diagram of an electronic device 100 is shown.

[0294] The electronic device 100 may be equipped with Or a portable terminal device of other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, a vehicle-mounted device, a smart home device and / or a smart city device, but not limited thereto, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or a touch panel, a desktop computer with a touch-sensitive surface or a touch panel, and other non-portable terminal devices, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0295] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0296] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0297] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0298] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0299] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0300] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0301] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0302] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0303] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0304] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0305] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0306] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0307] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0308] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0309] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0310] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0311] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0312] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0313] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0314] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0315] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0316] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0317] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0318] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0319] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0320] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0321] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0322] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0323] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0324] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0325] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0326] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0327] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0328] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0329] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0330] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0331] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0332] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0333] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0334] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0335] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0336] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0337] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0338] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0339] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0340] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0341] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0342] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0343] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0344] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0345] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0346] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0347] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0348] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0349] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0350] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0351] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0352] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0353] Fig.11 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0354] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0355] The application layer can include a series of application packages.

[0356] like Fig.11 As shown, the application package may include video editing, camera, gallery and other applications. The video editing application includes a user operation module, a data control module (operation), a data storage module and a data rendering module.

[0357] Specifically, the user operation module is used to receive various operations of the user, such as detecting an operation of adding a key frame, detecting an operation of playing, and the like.

[0358] The data control module is used to convert user operations into data that can be transmitted between various modules.

[0359] The data storage module includes a data transmission layer, a data storage module layer and a key frame data processing layer, wherein the data transmission layer is used to receive data sent by other modules and transmit the data to the data storage layer and the key frame data processing layer (also known as the data processing layer); the data storage layer includes a data monitor (visitor) and a material data class (Material), wherein the data monitor is used to monitor and identify the data sent to the data storage layer, and the material data class is used to store the parameters of various pictures and the content information of various pictures, such as the timestamp of the key frame and the parameters of each key frame picture; the key frame data processing layer is used to calculate the parameters of other pictures in the video or material according to the key frame picture parameters, and send the parameters of other pictures to the data rendering module.

[0360] The data rendering module includes a data rendering layer and Animation, wherein the Animation is used to render and generate the pictures displayed in the user interface in real time according to the parameters of various pictures and the contents of various pictures.

[0361] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0362] like Fig.11 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0363] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0364] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0365] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0366] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0367] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0368] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0369] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0370] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0371] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0372] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0373] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0374] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0375] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0376] A 2D graphics engine is a drawing engine for 2D drawings.

[0377] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0378] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0379] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0380] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0381] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0382] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0383] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video editing method, characterized in that: The method comprises: Displaying a first user interface, the first user interface comprising: a preview area, a first track, a second track, and a pointer at a fixed position, the first track displays multiple frames of a first video, the second track displays multiple frames of a first material, the preview area displays a first frame of the first track pointed to by the pointer and a second frame of the second track pointed to by the pointer, and each frame of the first material is presented as a first effect; receiving an operation of selecting the second track; Receive the operation of adding a keyframe; receiving a left sliding operation on the second track, moving the first track and the second track to the left, and displaying in the preview area a third picture of the first track after the pointer moves to the left and a fourth picture of the second track after the pointer moves to the left; receiving an editing operation acting on the fourth picture in the preview area, and adjusting the effect of the fourth picture in the preview area to the second effect; receiving a right sliding operation on the second track, moving the first track and the second track to the right, and displaying in the preview area a fifth picture of the first track after the pointer moves to the right and a sixth picture of the second track after the pointer moves to the right; Upon receiving a play operation, the first video is played starting from the fifth screen in the preview area, and the second material is played starting from the sixth screen, wherein the second material includes each screen of the first material, and in the second material, multiple screens before the second screen are presented as the first effect, multiple screens from the second screen to the fourth screen are gradually transitioned from the first effect to the second effect, and multiple screens after the fourth screen are presented as the second effect.

2. The method according to claim 1, characterized in that The second track is a picture-in-picture track, the first material is a video containing multiple frames, and in the preview area, the frames of the first material are displayed in the frame of the first video in a picture-in-picture format; or The second track is a text track, and the first material is a video containing multiple frames of text images; or The second track is a sticker track, and the first material is a video containing multiple frames of sticker images.

3. The method according to claim 1 or 2, characterized in that: The first effect includes displaying at a first display position, the editing operation is used to adjust the image from the first display position to a second display position, and the second effect includes displaying at the second display position; and / or The first effect includes displaying the image in a first size, the editing operation is used to adjust the image from the first size to a second size, and the second effect includes displaying the image in the second size; and / or The first effect includes displaying at a first display angle, the editing operation is used to adjust the image from the first display angle to a second display angle, and the second effect includes displaying at the second display angle; and / or The first effect includes displaying at a first opacity, the editing operation is used to adjust the image from the first opacity to a second opacity, and the second effect includes displaying at the second opacity.

4. The method according to any one of claims 1 to 3, characterized in that After receiving the operation of adding a key frame, the method further includes: displaying a first mark at the position of the second track pointed by the pointer; After receiving an editing operation acting on the fourth picture in the preview area, the method further includes: displaying a second mark at a position of the second track pointed to by the pointer after being moved to the left.

5. The method according to any one of claims 1 to 4, characterized in that After receiving the operation of adding a key frame, the method further includes: acquiring a parameter of the second picture, the parameter of the second picture being used to indicate the first effect; After receiving the editing operation acting on the fourth picture in the preview area, the method further includes: acquiring a parameter of the fifth picture, wherein the parameter of the fifth picture is used to indicate the second effect.

6. The method according to any one of claims 1 to 5, characterized in that Before displaying the first user interface, the method further includes: displaying a second user interface, wherein the second user interface includes: the preview area, the first track, and the pointer with a fixed position; An operation to add the second track is received.

7. The method according to any one of claims 1 to 6, characterized in that After the first video is played from the sixth screen in the preview area and the second material is played from the seventh screen, the method further includes: Receiving a user operation for exporting a video; A video including the first video and the second material is stored.

8. The method according to any one of claims 1 to 7, characterized in that In the preview area, the picture of the first material is overlaid on the picture of the first video, or the picture of the first material and the picture of the first video are displayed in split screen.

9. The method according to any one of claims 1 to 8, characterized in that The first track displays all or part of the first video.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: The start time of the first track is consistent with the start time of the second track.

11. The method according to any one of claims 1 to 10, characterized in that After receiving an operation of selecting the second track, the method further includes: A first control is displayed in the first user interface, and the operation of adding a key frame includes an operation acting on the first control.

12. The method according to any one of claims 1 to 11, characterized in that After the first video is played from the fifth screen in the preview area and the second material is played from the sixth screen, the method further includes: receiving a playback pause operation, and displaying the seventh frame of the first track pointed to by the pointer and the eighth frame of the second track pointed to by the pointer in the preview area; receiving an editing operation acting on an eighth screen in the preview area, adjusting the effect of the eighth interface in the preview area to a third effect, the eighth screen being located between the second interface and the fourth screen; receiving a right sliding operation on the second track, moving the first track and the second track to the right, and displaying in the preview area a ninth picture of the first track after the pointer points to the right and a tenth picture of the second track after the pointer points to the right; Upon receiving a play operation, the first video is played from the ninth screen in the preview area, and the third material is played from the tenth screen, wherein the third material includes screens of the first material arranged in sequence, and wherein multiple screens before the second screen are presented as the first effect, multiple screens from the second screen to the eighth screen are gradually transitioned from the first effect to the third effect, multiple screens from the eighth screen to the fourth screen are gradually transitioned from the third effect to the second effect, and multiple screens after the fourth screen are presented as the second effect.

13. The method according to any one of claims 1 to 12, characterized in that After receiving the operation of selecting the second track, the method further includes: A left handle is displayed at the left edge of the second track, and a right handle is displayed at the right edge of the second track. The left handle is used to adjust the start time of the second track, and the right handle is used to adjust the end time of the second track.

14. An electronic device, characterized in that: The electronic device includes a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in any one of claims 1-13.

15. A chip, applied to electronic equipment, characterized in that: The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method as described in any one of claims 1-13.

16. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 13.

Images