Live photo editing method and device, equipment, medium and product

Abstract

The invention discloses a live photo editing method, device and equipment, a medium and a product, and belongs to the technical field of multimedia. The method comprises the steps that in response to a selection operation for at least two live photos, an editing interface for the at least two live photos is displayed, and the editing interface comprises a selection area of at least one candidate audio; in response to a selection operation for a target audio on the editing interface, adding different audio clips of the target audio to the at least two live photos to obtain at least two edited live photos, the target audio being one of the at least one candidate audio; and in response to a preview operation for a first live photo in the at least two edited live photos, displaying a preview of the first live photo. According to the method, the different audio clips of the target audio can be quickly added to the at least two live photos respectively, a new live photo editing mode is provided, and the editing efficiency of the at least two live photos is improved at the same time.

Description

Technical Field

[0001] This application relates to the field of multimedia technology, and in particular to a method, apparatus, device, medium and product for editing live photos. Background Technology

[0002] Live photos are images that include sound and moving images. Related technologies support editing features such as adding filters and audio to live photos.

[0003] However, the editing methods in these technologies are rather rigid and cannot meet the diverse editing needs of users. Summary of the Invention

[0004] This application provides a method, apparatus, device, medium, and product for editing live photos, the technical solution of which is as follows:

[0005] According to one aspect of this application, a method for editing live photos is provided, the method comprising:

[0006] In response to a selection operation for at least two live photos, an editing interface for the at least two live photos is displayed, the editing interface including a selection area for at least one candidate audio.

[0007] In response to the selection operation for the target audio in the editing interface, different audio segments of the target audio are added to the at least two live photos respectively to obtain at least two edited live photos, wherein the target audio is one of the at least one candidate audio;

[0008] In response to a preview operation on the first of the at least two edited live photos, a preview of the first live photo is displayed.

[0009] According to one aspect of this application, a live photo editing apparatus is provided, the apparatus comprising:

[0010] A display module is configured to display an editing interface for the at least two live photos in response to a selection operation for at least two live photos, the editing interface including a selection area for at least one candidate audio.

[0011] An editing module is configured to, in response to a selection operation for a target audio in the editing interface, add different audio segments of the target audio to the at least two live photos respectively to obtain at least two edited live photos, wherein the target audio is one of the at least one candidate audio;

[0012] A preview module is configured to display a preview of the first live photo in response to a preview operation on the first live photo of the at least two edited live photos.

[0013] According to one aspect of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement a live photo editing method.

[0014] According to one aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements a live photo editing method.

[0015] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform a live photo editing method.

[0016] The beneficial effects of the technical solution provided in this application include at least the following:

[0017] This paper demonstrates a method for editing live photos, which allows the use of a target audio to concatenate at least two live photos. On one hand, it quickly completes the editing of at least two live photos, improving editing efficiency. On the other hand, the method is simple to operate, requiring only the selection of at least two live photos and a target audio, thus requiring minimal user input and improving human-computer interaction efficiency.

[0018] On the other hand, adding different audio segments of a target audio to at least two live photos, giving them a similar musical style, enhances their relevance and makes it easier to highlight their common theme, thus improving the artistry of the live photo editing process. Furthermore, for ordinary users (such as novice multimedia editors), it allows for simple editing to obtain live photos with clear themes and prominent subjects, reducing the difficulty of live photo editing. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown;

[0021] Figure 2 A flowchart illustrating a live photo editing method provided in an exemplary embodiment of this application is shown;

[0022] Figure 3 A flowchart illustrating a live photo editing method provided in an exemplary embodiment of this application is shown;

[0023] Figure 4 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0024] Figure 5 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0025] Figure 6 A flowchart illustrating a live photo editing method provided in an exemplary embodiment of this application is shown;

[0026] Figure 7 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0027] Figure 8 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0028] Figure 9 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0029] Figure 10 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0030] Figure 11 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0031] Figure 12 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0032] Figure 13 A schematic diagram of a live photo editing method provided in an exemplary embodiment of this application is shown;

[0033] Figure 14 This invention illustrates a structural block diagram of a live photo editing apparatus provided in an exemplary embodiment of this application;

[0034] Figure 15 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the settings and operation information involved in this application were obtained with full authorization.

[0039] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0040] First, let me introduce the relevant terms used in this application:

[0041] Live photo: An image that includes sound and moving visuals. Live photos come in both static and dynamic forms. Normally, a live photo is static, meaning it's an image without sound or moving visuals. Users can trigger a live photo to transform from static to dynamic by long-pressing or clicking. A dynamic live photo can be understood as a video, accompanied by sound and moving visuals; in other words, a dynamic live photo is an image that plays sound and moving visuals. Optionally, a static live photo can be called a cover image, which is a frame from the dynamic visuals of the live photo. This cover image can be selected by the user, set by the developer, or determined by the terminal according to preset rules. For example, the developer sets the first frame of the dynamic visuals as the cover image; or the terminal determines the frame at the moment the shutter is pressed as the cover image.

[0042] A feed refers to a data format that continuously updates and presents social content to users. Feeds include at least two display formats: timeline-based and random. Timeline-based display shows social content to users in the order it was updated. Random display calculates the weight of content based on certain factors to determine the order in which it is shown. For example, an algorithm might calculate the weight of social content published by a subscription account, and then use that weight to determine the order in which it is recommended to users. Feeds also use different presentation formats to display social content, such as list, waterfall, and card formats. A feed can also be considered as an aggregation of at least one resource provided by at least one content source subscribed to by a user's account.

[0043] Figure 1 An architectural diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system includes a terminal 110.

[0044] Terminal 110 can be a laptop computer, desktop computer, mobile phone, tablet computer, e-book reader, and video game console, etc.

[0045] Terminal 110 includes a memory and a processor; the memory may include one or more computer-readable storage media. The computer-readable storage media include at least one of random access memory (RAM), read-only memory (ROM), and flash memory. An operating system and applications are installed in the memory.

[0046] An operating system is the foundational software that provides applications with secure access to computer hardware. An operating system can be either Android or iOS. The operating system supports the download and installation of applications.

[0047] Optionally, terminal 110 also includes a touchscreen; the touchscreen can be a capacitive screen or a resistive screen. The touchscreen is used to realize the interaction between terminal 110 and user 120. In the embodiments of this application, terminal 110 obtains interactive operations on the multimedia processing client triggered by user 120 through the touchscreen.

[0048] Optionally, the multimedia processing client supports importing, editing, and exporting live photos. Optionally, the multimedia processing client also supports converting other forms of multimedia files such as images and videos into live photos. Furthermore, the multimedia processing client supports performing other processing on live photos, such as adding effects, changing the cover of the live photo, and adding virtual stickers to the live photo. This application embodiment does not limit these aspects.

[0049] Optionally, the computer system also includes a server 130. Server 130 can be any of multiple servers, virtual cloud storage, or a cloud computing center. Optionally, server 130 is used to provide terminal 110 with an installation package for a multimedia processing client; or, server 130 is used to provide terminal 110 with target audio; or, server 130 is used to provide terminal 110 with special effects, virtual stickers, etc., from the multimedia processing client.

[0050] Terminal 110 has a multimedia processing client installed and running. Optionally, the multimedia processing client is designed with at least one of a homepage 140 and a selection interface 150. The homepage 140 is used to display the functions of the multimedia processing client, such as editing, shooting, and the audio concatenation function provided in this embodiment. The selection interface 150 is used to select at least two live photos.

[0051] Optionally, the multimedia processing client is designed with a first editing interface 160 and / or a second editing interface 170. The first editing interface 160 and / or the second editing interface 170 are used to edit at least two live photos, such as adding different audio segments from a target audio to each of the at least two live photos.

[0052] Terminal 110 is the terminal used by user 120. User 120 can operate the first editing interface 160 and / or the second editing interface 170 to edit at least two live photos to obtain at least two edited live photos. The operation performed by user 120 on at least one of the aforementioned homepage 140, selection interface 150, first editing interface 160, and second editing interface 170 is called an interactive operation. Optionally, terminal 110 changes the content displayed on at least one of the homepage 140, selection interface 150, first editing interface 160, and second editing interface 170 according to the interactive operation; or, terminal 110 adapts the content displayed by the multimedia processing client according to the interactive operation.

[0053] In some embodiments, the terminal 110 and the server 130 are connected via a wired or wireless network.

[0054] For example, user 120 on homepage 140 instructs to use the audio concatenation function by clicking audio concatenation control 141. In response to the user's triggering action on audio concatenation control 141, selection interface 150 is displayed, which instructs the user to select at least two live photos. Selection interface 150 displays multiple candidate live photos, and the user can click on a candidate live photo to view and select that live photo as at least two live photos to be edited.

[0055] For example, in response to a user's selection of at least two live photos, such as after the user selects at least two live photos on the selection interface 150 and clicks the editing control 151, an editing interface (such as the first editing interface 160 or the second editing interface 170) is displayed.

[0056] Optionally, in response to a user's selection of at least two live photos, a first editing interface 160 is displayed. The first editing interface displays a feed preview area 161, which indicates the preview effect of the edited at least two live photos after they are published to the feed interface. Upon entering the first editing interface 160, the first editing interface 160 displays a preview effect of the selected at least two live photos after they are published to the feed interface. Optionally, the multimedia processing client supports previewing the effect of publishing the edited at least two live photos to the feed interface of the information interaction platform. The first editing interface 160 also includes an audio selection area 162, which allows the user to select a target audio from at least two candidate audios. In response to the user's selection of the target audio, different audio segments of the target audio are added to each of the at least two live photos to obtain the edited at least two live photos. Optionally, the first editing interface 160 also supports users playing the first live photo by triggering an action on the first live photo among at least two live photos. That is, the information stream preview area 161 displays at least two static live photos, or rather, the covers of at least two live photos. Users can preview dynamic live photos by triggering actions, that is, playing the dynamic visuals and sound of the live photo. For the edited live photo, the sound played will include the audio segment corresponding to that live photo. This achieves the goal of linking at least two live photos with a single target audio.

[0057] Optionally, in response to a user's selection of at least two live photos, a second editing interface 170 is displayed. The second editing interface 170 displays editing operations on the at least two live photos. The second editing interface 170 also supports triggering an operation on the first live photo among the at least two live photos to play the first live photo.

[0058] Optionally, the layout of the information flow preview area 161 in the first editing interface 160 is the same as or similar to the information flow interface in the social interaction platform. The social interaction platform is used to provide social functions for user 120, including at least one of the social content displayed on the information flow interface and browsing social content on the information flow interface. That is, the information flow preview area 161 is used to display a preview of at least two edited live photos as social content published to the information flow interface.

[0059] Optionally, the social interaction platform and the multimedia processing client are different clients; or, the social interaction platform and the multimedia processing client belong to the same client. That is, there is one client that supports executing the methods provided in one or more exemplary embodiments of this application; or, there are two associated clients, wherein the multimedia processing client supports invoking, waking up, or requesting the display of the information flow publishing interface of the social interaction platform.

[0060] Figure 2 A flowchart illustrating a live photo editing method provided in an exemplary embodiment of this application is shown. The method is executed by a terminal, which may be... Figure 1 The terminal 110 shown can also be executed by a multimedia processing client, which can be installed on... Figure 1 The software client in terminal 110 shown. The method includes:

[0061] Step 210: In response to a selection operation for at least two live photos, display an editing interface for at least two live photos, the editing interface including a selection area for at least one candidate audio.

[0062] In some embodiments, the aforementioned live photos can also be replaced with videos; that is, the live photo editing method shown in the embodiments of this application is also applicable to video editing. The embodiments of this application will use live photos as examples, but are not intended to limit the scope of the application.

[0063] Optionally, the live photo is obtained by directly taking a picture from the terminal; or, the live photo is obtained by converting it through a software client (such as the multimedia processing client shown in the embodiments of this application), that is, converting pictures or videos into live photos.

[0064] Optionally, the multimedia processing client includes a selection interface that displays at least one of live photos, videos, and images stored in the terminal and / or the multimedia processing client. The user selects at least two live photos by triggering an action in the selection interface. The triggering action includes at least one of the following: a click, a long press, a swipe, a touch, a button press, a gesture, a voice control, an action control, and any combination of at least two of the above actions.

[0065] Optionally, the selection interface can also be called the editing interface. That is, the selection operation for at least two live photos is also part of the editing operation for at least two live photos. In other words, in some optional embodiments, Figure 1 The selection interface 150 shown in the image and the first editing interface 160 both belong to the editing interface for at least two live photos.

[0066] Keypad operation refers to triggering the live photo request control via physical buttons on a keyboard, mouse, gamepad, etc. Gesture operation refers to a combination of operations such as swiping, clicking, and long-pressing with one or more fingers on a touchscreen or touchpad, such as pinching with two fingers or swiping down with three fingers. It should be noted that gesture operation can also be air gesture, where the user's hand movements are captured by the terminal's camera. Voice control operation refers to triggering the live photo request control via voice commands, such as saying "Select the second live photo in the first row" or "Select the fourth live photo in the third row." Motion control operation refers to selecting live photos based on the user's actions. This involves capturing or recognizing the user's actions through the terminal's gyroscope, motion sensors attached to the terminal, or actions detected by the terminal's camera, thereby determining the desired live photo based on the user's movements.

[0067] Step 220: In response to the selection operation for the target audio in the editing interface, add different audio segments of the target audio to at least two live photos to obtain at least two edited live photos, where the target audio is one of at least one candidate audio.

[0068] Optionally, after entering the editing interface, an audio will be automatically selected as the target audio; and / or, an audio selection area (i.e., a selection area for at least one candidate audio) is displayed in the editing interface, and the audio selection area displays at least one audio selection control. The user determines the candidate audio corresponding to the audio selection control as the target audio by triggering the audio selection control, that is, the target audio is one of at least one candidate audio.

[0069] Optionally, adding different audio segments of the target audio to at least two live photos means that the target audio has multiple audio segments, and matching each of the at least two live photos with one of the multiple audio segments of the target audio to obtain the audio segment corresponding to each of the at least two live photos. Optionally, the audio segments corresponding to different live photos are also different.

[0070] Optionally, at least two live photos are live photos containing audio, or at least two live photos are live photos without audio. When at least two live photos contain audio, adding different audio segments of the target audio to each of the at least two live photos includes the following two possible scenarios: first, removing the audio present in the at least two live photos, and then adding different audio segments of the target audio to the at least two live photos after audio removal; or directly adding different audio segments of the target audio to the at least two live photos.

[0071] In other words, at least two live-action photos with different audio segments of the target audio added, or live-action photos with added audio segments, or edited live-action photos, differ from the original live-action photos in two ways: either only the added audio segments are present, or the added audio segments and the original audio of the live-action photos are present. Depending on the method of adding the target audio used in the actual implementation, the final effect of the at least two edited live-action photos will also be different. This application does not limit the method of adding audio, but the scope of protection is not limited thereto.

[0072] Step 230: In response to a preview operation for the first of at least two edited Live Photos, display a preview of the first Live Photo.

[0073] Optionally, in response to a preview operation on the first live photo among at least two edited live photos in the editing interface, a preview of the first live photo is displayed in the editing interface; or, in response to a preview operation on the first live photo among at least two edited live photos in the editing interface, a preview interface is displayed, and a preview of the first live photo is displayed in the preview interface.

[0074] The preview of the first live photo is the playback of the first live photo. Or, it is the playback of the dynamic image and sound of the first live photo (i.e., an audio clip, or an overlay of an audio clip and the original audio).

[0075] In summary, the method provided in this application embodiment demonstrates a method for editing live photos, which can achieve the purpose of using a target audio to concatenate at least two live photos. On the one hand, it quickly completes the editing operation of at least two live photos, improving the efficiency of live photo editing; on the other hand, this method is simple to operate, requiring only the selection of at least two live photos and a target audio, which does not require high user operation skills and improves human-computer interaction efficiency.

[0076] On the other hand, adding different audio segments of a target audio to at least two live photos, giving them a similar musical style, enhances their relevance and makes it easier to highlight their common theme, thus improving the artistry of the live photo editing process. Furthermore, for ordinary users (such as novice multimedia editors), it allows for simple editing to obtain live photos with clear themes and prominent subjects, reducing the difficulty of live photo editing.

[0077] • Set up the information flow preview area.

[0078] In some embodiments, the aforementioned multimedia processing client supports sharing at least two edited Live Photos to a social interaction platform, which has a feed interface. The feed interface includes social content posted by at least one content-publishing account, arranged chronologically. The user's logged-in account on the social interaction platform running on the terminal, as well as the corresponding friend accounts, can all serve as content-publishing accounts to post social content, which will be displayed on the user's account's feed interface. That is, at least two edited Live Photos can be posted to the feed interface. Therefore, the following methods can be used to improve the efficiency of users posting at least two edited Live Photos to the feed interface. Based on... Figure 2 In an alternative embodiment, such as Figure 3 As shown, the method also includes:

[0079] Step 310: Display the feed preview area in the editing interface. The feed preview area is used to indicate the preview effect of at least two edited Live Photos after they are published to the feed interface.

[0080] Optionally, in response to a selection operation on at least two live photos, an editing interface is displayed, which includes a feed preview area. Optionally, the display of the feed preview area is unconditional, meaning it is displayed only in the edit interface; or, the display of the feed preview area is conditional, appearing in response to a feed preview operation in the editing interface. Feed preview operations include, but are not limited to, click operations, long press operations, swipe operations, touch operations, button operations, gesture operations, voice control operations, motion control operations, and free combinations of at least two of the above operations. For a detailed description of the above operation methods, please refer to the trigger operations shown in the selection interface above; further details will not be repeated here.

[0081] Optionally, the feed preview area is used to indicate the preview effect of at least two edited Live Photos after they are published to the feed interface. When the display of the feed preview area is unconditional, the Live Photos displayed in the feed preview area can also be at least two unedited Live Photos. The feed preview area also supports displaying only one Live Photo; this embodiment of the application does not limit this.

[0082] Optionally, the preview effect displayed in the feed preview area is consistent with the preview effect on the social interaction platform; the feed preview area also supports the same preview method as the social interaction platform. That is, the feed preview area allows users to view each of at least two live photos through the feed preview operation, and also supports triggering the playback of live photos. Playing live photos can be understood as displaying the dynamic picture corresponding to the live photo and playing the audio clip corresponding to the live photo.

[0083] Optionally, the feed preview area displays at least two edited live photos. Optionally, the at least two live photos displayed in the feed preview area are marked with a live icon, which indicates that the image is a live photo.

[0084] Optionally, the editing interface also displays a feed toggle control, updating the feed preview area's layout in response to triggering the feed toggle control. That is, the editing interface supports previewing the effect of at least two edited Live Photos published on multiple social media platforms or multiple feed interfaces. The layout of multiple Live Photos published on different social media platforms or different feed interfaces may differ; users can update the feed preview area's layout by clicking the feed toggle control.

[0085] In some embodiments, step 310 can be performed simultaneously with step 210 above, that is, when the display of the information flow preview area is unconditional, the information flow preview area located in the editing interface is displayed at the same time as the editing interface is displayed; step 310 can be performed after step 210 above, that is, when the display of the information flow preview area is conditional, the display of the information flow preview area can be triggered by the user's information flow preview operation in the editing interface.

[0086] In summary, the method provided in this application displays a preview area in the editing interface, directly showing the effect of at least two edited live photos published to the live feed interface on the multimedia processing client. This eliminates the need for users to first save the edited photos to their album, then open the social interaction platform and sequentially add the edited live photos one by one to preview the publishing effect on the live feed interface. If the user is not satisfied with the publishing effect, they need to reopen the multimedia processing client, reselect at least two live photos, select the target audio to obtain the processed at least two live photos, and then save and reopen the social interaction platform to preview them again. This method supports direct previewing of the effect published to the live feed interface of the social interaction platform within the editing interface, improving the efficiency of users editing at least two live photos, thereby indirectly improving the efficiency of users publishing the edited at least two live photos to the live feed interface. Furthermore, it eliminates the need to frequently save unsatisfactory edited live photos, saving terminal storage space.

[0087] (1) Preview of the first live photo.

[0088] In some embodiments, when the editing interface includes a feed preview area, previewing at least two edited live photos displayed in the feed preview area will have a similar browsing (or viewing) effect to posting them to the feed interface. That is, step 230 above can be implemented as follows: in response to a triggering operation on the first live photo among the at least two edited live photos in the feed preview area, the first live photo is played, and the audio of the first live photo includes an audio segment corresponding to the first live photo.

[0089] The triggering operations for the first live photo among at least two edited live photos include, but are not limited to, click operations, long press operations, swipe operations, touch operations, button operations, gesture operations, voice control operations, motion control operations, and free combinations of at least two of the above operations. For a detailed description of the above operation methods, please refer to the triggering operations shown in the selection interface above; they will not be repeated here.

[0090] Optionally, the triggering operation for the first live photo among the at least two edited live photos can be described as: a dynamic triggering operation for the first live photo among the at least two edited live photos, that is, converting the static first live photo into a dynamic one.

[0091] Optionally, before performing the triggering operation on the first live photo among the at least two edited live photos, a selection operation on the first live photo among the at least two edited live photos may also be performed. In response to the selection operation on the first live photo among the at least two edited live photos, a preview interface of the first live photo is displayed.

[0092] Optionally, the triggering operation for the first of the at least two edited live photos can be performed in the feed preview area or in the preview interface.

[0093] Optionally, the display area of ​​the first live photo in the preview interface is larger than the display area of ​​the first live photo in the information stream preview area. That is, by selecting the first live photo among at least two edited live photos, the first live photo is highlighted, or in other words, the first live photo is enlarged.

[0094] For example, such as Figure 4As shown in section (1) of the document, an information flow preview area 51 is displayed in the editing interface 50. At least two live photos are displayed in the information flow preview area 51, and the display area of ​​each of the at least two live photos displayed in the information flow preview area 51 is a first area. Optionally, a live photo identifier 53 is displayed on each of the at least two live photos, indicating that the image is a live photo. In response to a user's selection operation in the information flow preview area 51 for the first live photo 52 among the edited at least two live photos, a preview interface 54 including the first live photo 52 is displayed. For example, after the user clicks on the first live photo 52 in the information flow preview area 51, a preview interface 54 including the first live photo 52 is displayed. The preview interface 54 is as follows... Figure 4 As shown in section (2), the display area of ​​the first live photo 52 displayed on the preview interface 54 is the second area, which is larger than the first area. In response to a trigger operation on the first live photo 52, the first live photo 52 is played on the preview interface 54. As in Figure 4 In the preview interface 54 shown in part (2), the user long-presses the first live photo 52. When the user's pressing duration reaches the pressing duration threshold, the first live photo 52 is played. The first live photo 52 includes not only dynamic visuals but also audio during playback. The audio of the first live photo 52 includes the audio segment corresponding to the first live photo 52, or in other words, the playback of the first live photo 52 is accompanied by the playback of the corresponding audio segment. Optionally, the preview interface 54 also displays a preview indicator area 55, which is used to indicate the position (or order) of the currently previewed live photo among at least two live photos and / or the number of at least two live photos, such as... Figure 4 The preview indicator area 55 in the preview interface 54 shown in part (2) indicates that there are at least two live photos, which are nine live photos. The current preview is the first of the nine live photos.

[0095] In summary, the method provided in this application embodiment demonstrates a method for previewing the first live photo among at least two edited live photos in the information flow preview area. It supports directly previewing the effect of the information flow interface published to the social interaction platform in the editing interface, improves the efficiency of users editing at least two live photos, and eliminates the need to frequently save at least two edited live photos that do not meet the requirements, thus saving terminal storage space.

[0096] (2) Preview of the second live photo.

[0097] In some embodiments, the information flow interface typically also supports continuous previewing of multimedia files. Therefore, to enable users to preview the first live photo on the multimedia processing client while simultaneously changing the previewed live photo in a manner consistent with the information flow interface, the method further includes: in response to a triggering operation on the second live photo or after the first live photo has finished playing, playing the second live photo, the second live photo being the next live photo adjacent to the first live photo, the audio of the second live photo including the audio segment corresponding to the second live photo, and the audio segment corresponding to the second live photo being continuous with the audio segment corresponding to the first live photo.

[0098] Optionally, the triggering operation for the second live photo is similar to the triggering operation for the first live photo among the at least two edited live photos in the information stream preview area, or the triggering operation for the second live photo includes: a switching operation for at least two live photos, or a switching triggering operation for the first live photo.

[0099] Optionally, the triggering operation for the second live photo can be triggered in the information stream preview area or in the preview interface.

[0100] For example, such as Figure 4 As shown in part (1), an information stream preview area 51 is displayed in the editing interface 50, and a second live photo 56 is displayed in the information stream preview area 51. In response to the user's selection operation on the second live photo 56 (such as clicking on the second live photo 56 in the information stream preview area 51), a preview interface 54 including the second live photo 56 is displayed, as shown in part (1). Figure 4 As shown in section (3) of the document. In response to a trigger operation on the second live photo 56, the second live photo 56 is played on the preview interface 54. As shown in Figure 4 In the preview interface 54 shown in part (3), the user long-presses the second live photo 56. When the user's pressing duration reaches the pressing duration threshold, the second live photo 56 is played. The second live photo 56 includes not only dynamic images but also audio during playback. The audio of the second live photo 56 includes the audio segment corresponding to the second live photo 56, or in other words, the playback of the second live photo 56 is accompanied by the playback of the audio segment corresponding to the second live photo 56.

[0101] For example, such as Figure 4 As shown in part (2), a first live photo 52 is displayed in the preview interface 54. In response to a trigger operation on the second live photo 56 (such as swiping left or right in the preview interface 54), the second live photo 56 is displayed and played in the preview interface 54, or in other words, the first live photo 52 displayed in the preview interface 54 is switched to the second live photo 56 and the second live photo 56 is played. Figure 4 As shown in section (3) of the document. Or, as... Figure 4 As shown in part (2), a first live photo 52 is displayed in the preview interface 54. In response to a first trigger operation on the second live photo 56 (such as swiping left or right in the preview interface 54), the second live photo 56 is displayed in the preview interface 54, or the first live photo 52 displayed in the preview interface 54 is switched to the second live photo 56. In response to a second trigger operation on the second live photo 56 (such as pressing the second live photo for a duration that reaches a threshold), the second live photo 56 is played.

[0102] Optionally, in response to a switching trigger operation from the first live photo to the second live photo, the display of the second live photo is switched and the second live photo is played. Alternatively, in response to a switching trigger operation from the first live photo to the second live photo, the display of the second live photo is switched; and in response to a playback trigger operation for the second live photo, the second live photo is played. That is, after the user triggers the switch from the first live photo to the second live photo, the second live photo can be played automatically while it is being displayed; or, after the user triggers the switch from the first live photo to the second live photo, only the static second live photo is displayed, and the dynamic form of the second live photo requires an additional trigger operation.

[0103] Specifically, when a user triggers a switch from the first live photo to the second live photo, the second live photo can be automatically played while it is being displayed. Since the first and second live photos are adjacent, their corresponding audio segments are also continuous. Switching to the second live photo ensures that the corresponding audio segment will play automatically after the switch. This allows for the continuous connection of continuous audio segments of the target audio during the switch from the first to the second live photo, achieving the goal of connecting at least two live photos with a single target audio. This enhances the thematic relevance and interactivity of the edited at least two live photos, improves their interactivity, and increases their fun and interactivity.

[0104] Optionally, after the first live photo finishes playing, the first live photo is replayed; or, after the first live photo finishes playing, a static version of the first live photo is displayed; or, after the first live photo finishes playing, the display and playback of the second live photo automatically switches to that of the first live photo, where the second live photo is the next live photo after the first live photo, or the audio segment corresponding to the second live photo is the next audio segment following the audio segment corresponding to the first live photo. That is, when the user triggers the playback of the first live photo, after the first live photo finishes playing, the system automatically switches to and plays the second live photo, making the process of browsing and editing at least two live photos smoother and more fluid for the user.

[0105] For example, the way live photos are played is as follows: Figure 5 As shown, taking the first live photo as an example, after the first live photo is triggered to play, the static first live photo (i.e., the cover of the first live photo) is played first, in a silent state. Then, the dynamic first live photo (i.e., the dynamic picture and sound of the first live photo, such as dynamic picture and audio clips) is played. After the dynamic picture finishes playing, the static first live photo is displayed again. The above processes from static to dynamic and from dynamic to static are enhanced with a "dissolve transition" effect to make the transitions smoother.

[0106] In summary, the method provided in this application embodiment demonstrates the preview effect of the second live photo, supports direct preview of the effect published to the information flow interface of the social interaction platform in the editing interface, improves the efficiency of users editing at least two live photos, and eliminates the need to frequently save at least two edited live photos that do not meet the requirements, thus saving the terminal's storage space.

[0107] Furthermore, when the second live photo is adjacent to the first live photo, the audio segment of the second live photo is continuous with the audio segment of the first live photo, thus achieving the goal of using a single target audio to string together at least two live photos. Adding different audio segments of a target audio to at least two live photos ensures that the edited at least two live photos have the same musical style, enhancing the relevance of the edited at least two live photos, making it easier to highlight the common theme of the edited at least two live photos, and improving the artistry of the live photo editing process. Moreover, for ordinary users (such as novice multimedia editors), it allows for simple editing to obtain live photos with a clear theme and prominent subject, reducing the difficulty of live photo editing.

[0108] On the other hand, the editing method shown in the embodiments of this application enables the concatenation of a target audio with at least two live photos without first segmenting and cropping the target audio into multiple audio segments, and then adding the cropped audio segments to each live photo separately, thus improving the editing efficiency of at least two live photos. Since editing supports simultaneous editing of at least two live photos rather than editing them one by one, the preview interface allows for viewing of at least two edited live photos on a single screen, rather than previewing one edited live photo and then exiting to select another, further improving the preview efficiency of edited live photos.

[0109] • Published on social media platforms.

[0110] In some embodiments, after a user completes editing at least two live photos, the edited photos can be published to a social interaction platform, or to the information feed interface of the social interaction platform, via the aforementioned multimedia processing client. Optionally, the social interaction platform and the multimedia processing client are different clients; or, the social interaction platform and the multimedia processing client belong to the same client. That is, there is one client that supports executing the method provided in one or more exemplary embodiments of this application; or, there are two associated clients, wherein the multimedia processing client supports invoking, waking up, or requesting the display of the information feed publishing interface of the social interaction platform.

[0111] Optionally, the method further includes: displaying the edited at least two live photos in a feed interface in response to a publishing operation for at least two edited live photos.

[0112] Optionally, when the social interaction platform and the multimedia processing client are different clients, in response to a publishing operation for at least two edited live photos, the terminal or multimedia processing client sends a social content publishing request to the social interaction platform, the social content publishing request including at least two edited live photos. In response to receiving the social content publishing request, the social interaction platform displays a feed publishing interface, which displays at least two edited live photos; in response to a publishing operation on the feed publishing interface, it displays the feed interface and the at least two edited live photos.

[0113] Optionally, when the social interaction platform and the multimedia processing client belong to the same client, in response to the publishing operation for at least two edited live photos, a feed publishing interface is displayed, showing at least two edited live photos. Alternatively, in response to the publishing operation for at least two edited live photos, a feed interface is displayed, showing at least two edited live photos. Optionally, when the social interaction platform and the multimedia processing client belong to the same client, the feed publishing interface is the aforementioned editing interface displaying a feed preview area, such as... Figure 4 As shown in section (1) of the document; or, in response to a selection operation for at least two live photos, a feed publishing interface for at least two live photos is displayed, or a feed publishing interface is displayed, in which at least two live photos are shown. Optionally, the feed publishing interface also includes a selection area for at least one candidate audio and / or a feed preview area. In response to a selection operation for a target audio in the editing interface, different audio segments of the target audio are added to each of the at least two live photos to obtain edited at least two live photos.

[0114] Optionally, when the social interaction platform and the multimedia processing client belong to the same client, the method further includes: in response to a trigger operation on a first live photo among at least two edited live photos in the information flow interface, playing the first live photo, the audio of the first live photo including an audio segment corresponding to the first live photo; in response to a trigger operation on a second live photo or after the first live photo has finished playing, playing the second live photo, the second live photo being a live photo adjacent to the first live photo, the audio of the second live photo including an audio segment corresponding to the second live photo, the audio segment corresponding to the second live photo being continuous with the audio segment corresponding to the first live photo.

[0115] Optionally, when the social interaction platform and the multimedia processing client are different clients, the terminal or the social interaction platform, in response to a trigger operation on the first live photo among at least two edited live photos in the information flow interface, plays the first live photo, the audio of which includes an audio segment corresponding to the first live photo; in response to a trigger operation on the second live photo or after the first live photo has finished playing, plays the second live photo, which is a live photo adjacent to the first live photo, the audio of which includes an audio segment corresponding to the second live photo, and the audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo.

[0116] Optionally, in response to a trigger operation on the first of at least two edited live photos in the information flow interface, the first live photo is played, and the audio of the first live photo includes the audio segment corresponding to the first live photo. Similar to "(1) Preview of the first live photo" above, it will not be repeated here.

[0117] Optionally, in response to a trigger operation on the second live photo, the second live photo is played. The second live photo is an adjacent live photo to the first live photo. The audio of the second live photo includes an audio segment corresponding to the second live photo, and the audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo. Similar to "(2) Preview of the second live photo" above, it will not be repeated here.

[0118] In summary, the method provided in this application supports publishing at least two edited live photos by calling or waking up a social interaction platform through a multimedia processing client. This eliminates the need for users to exit the multimedia client, find and select the social interaction platform on the terminal's homepage, and then publish the edited at least two live photos through the social interaction platform. This simplifies the publishing process of at least two edited live photos and improves the efficiency of publishing at least two edited live photos.

[0119] • Save to album.

[0120] In some embodiments, users can save at least two edited Live Photos to their albums, in addition to publishing them to social interaction platforms as shown above.

[0121] Optionally, the method further includes: in response to an export operation for at least two edited Live Photos, exporting the at least two edited Live Photos to an album.

[0122] Optionally, the album is the system album of the terminal device, or the album is the software album of the multimedia processing client.

[0123] For example, such as Figure 4 As shown in section (1), the export operation for at least two edited live photos is a trigger operation for the save control 57, such as when the user clicks the save control 57. Figure 6 As shown in section (1), in response to an export operation for at least two edited Live Photos, a save progress interface 60 is displayed. The save progress interface 60 indicates the save progress of the currently edited at least two Live Photos. Optionally, the save progress interface displays a save progress area 61, which displays the current save progress, such as... Figure 6As shown in section (1), the current edited at least two live photos are 25%, and 3 / 9 indicates that the third photo in the edited at least two live photos is being processed.

[0124] Optionally, after at least two edited Live Photos have been saved, a save completion screen (62) is displayed, such as... Figure 6 As shown in section (2) of the document. The Save Complete screen 62 indicates that the saving of at least two edited live photos has been successfully completed. Optionally, the Save Complete screen 62 also displays a first publishing control 63, a second publishing control 64, and a third publishing control 65. For example, the first publishing control 63 is a publishing control corresponding to Moments, or a publishing control corresponding to a social interaction platform, or a publishing control of the information flow interface. In response to a trigger operation on the first publishing control 63, the above-mentioned "publish to social interaction platform" is executed. Optionally, the second publishing control 64 is a publishing control corresponding to Video Account, or a publishing control corresponding to a social interaction platform, or a publishing control of the information flow interface. In response to a trigger operation on the second publishing control 64, the at least two edited live photos are converted into a video and then published to the social interaction platform. Here, converting the at least two edited live photos into a video means that the video includes the at least two edited live photos. Optionally, the third publishing control 65 is a publishing control corresponding to other social interaction platforms, other information flow interfaces, or other chat clients.

[0125] In some embodiments, after a user saves at least two edited Live Photos to an album, the user can access the saved Live Photos in the album through the feed publishing interface. In response to a selection operation on the feed publishing interface or social interaction platform for a third Live Photo among the at least two edited Live Photos, the remaining Live Photos among the at least two edited Live Photos are automatically selected. The remaining Live Photos refer to at least one edited Live Photo other than the third Live Photo, which is any one of the at least two edited Live Photos. That is, when any one Live Photo among the at least two edited Live Photos is selected from the album or when the album is accessed, the remaining Live Photos among the at least two edited Live Photos are automatically selected; or, when multiple Live Photos among the at least two edited Live Photos are selected from the album or when the album is accessed, the remaining Live Photos among the at least two edited Live Photos are automatically selected, wherein the ratio of the multiple Live Photos to the at least two edited Live Photos satisfies an automatic selection threshold. In other words, users no longer need to individually select at least one edited Live Photo. The terminal or social interaction platform can determine the user's intent based on the currently selected Live Photo. If the user has selected any one or more Live Photos that belong to at least two edited Live Photos, the platform determines that the user's intent is to select the remaining unselected Live Photos from the at least two edited Live Photos. This method eliminates the need for the terminal or social interaction platform to determine the user's intent and eliminates the need for users to individually select at least two edited Live Photos, reducing user steps and improving the efficiency of users posting information on the feed interface.

[0126] Optionally, the above-mentioned automatically selected and edited at least two live photos have a first order.

[0127] Optionally, at least two edited live photos have the same or similar identification information, which indicates that multiple live photos were edited using the live photo editing method shown in the embodiments of this application, and that each corresponds to a target audio.

[0128] For example, an identifier is generated based on the user's trigger time for the save control, and this identifier is added to each of the at least two edited live photos; or, at least two identifiers are generated based on the user's trigger time for the save control, and an identifier is added to each of the at least two edited live photos.

[0129] In summary, the method provided in this application embodiment also supports users to save at least two edited live photos to their album, so that they can enjoy the edited at least two live photos at any time for remembrance, thus preserving the user's editing results and improving the reliability of human-computer interaction.

[0130] • Set up associations between live photos and audio clips.

[0131] For the process of matching different audio segments of a target audio to at least two live photos, adding a different audio segment of the target audio to at least two live photos can enhance the theme of the edited at least two live photos, but it is difficult to guarantee the coherence of the edited at least two live photos in the preview. Therefore, the following methods can be used to improve the coherence of the edited at least two live photos.

[0132] Figure 7 A flowchart illustrating a live photo editing method provided in an exemplary embodiment of this application is shown.

[0133] Step 221: In response to the selection operation for the target audio in the editing interface, add the j-th audio segment of the target audio to the i-th live photo among at least two live photos; add the j+1-th audio segment of the target audio to the (i+1)-th live photo among at least two live photos.

[0134] Where i and j are both positive integers; the j-th audio segment is consecutive to the (j+1)-th audio segment.

[0135] Optionally, the target audio includes multiple audio segments. These multiple audio segments are pre-trimmed by the developer; or, the multiple audio segments are trimmed by the server according to a set trimming method; or, the multiple audio segments are trimmed by the user; or, the multiple audio segments are trimmed by the terminal according to a set trimming method; or, the multiple audio segments are trimmed by the multimedia processing client according to a set trimming method.

[0136] Optionally, the target audio includes multiple audio segments that correspond one-to-one with at least two live photos; or, the target audio includes more than two audio segments. That is, the number of audio segments included in the target audio is greater than or equal to the number of live photos in the at least two live photos.

[0137] Optionally, in response to a selection operation for the target audio in the editing interface, consecutive audio segments from the target audio are added to adjacent live photos in at least two live photos. That is, the j-th audio segment from the target audio is added to the i-th live photo in at least two live photos, and the j-th audio segment from the target audio is added to the (i+1)-th live photo in at least two live photos. It should be noted that i and j can be equal or unequal.

[0138] For example, in cases where the number of audio segments in the target audio is greater than the number of live photos in at least two live photos, if the at least two live photos include n live photos and the target audio includes m audio segments, where n < m, then the first n audio segments can be matched with each of the n live photos; or, consecutive n audio segments from the m audio segments can be matched with each of the n live photos. For example, if n is 3 and m is 5, then the first n audio segments can be matched with each of the n live photos, that is, the first 3 audio segments of the 5 audio segments can be matched with each of the 3 live photos. For example, the first audio segment in the target audio can be matched with the first live photo in at least two live photos (i.e., the first audio segment of the target audio can be added to the first live photo), the second audio segment in the target audio can be matched with the second live photo in at least two live photos, and the third audio segment in the target audio can be matched with each of the at least two live photos. Match the third live photo in the photo; or, match n consecutive audio segments from m audio segments with n live photos, that is, match 3 consecutive audio segments from 5 audio segments with 3 live photos, such as matching the second audio segment in the target audio with the first live photo from at least two live photos, matching the third audio segment in the target audio with the second live photo from at least two live photos, and matching the fourth audio segment in the target audio with the third live photo from at least two live photos.

[0139] It should be noted that the order of the live photos in the aforementioned at least two live photos can be the order in which the user selects the at least two live photos, or the order in which the multimedia processing client, server, or terminal sorts them. This application embodiment does not limit this.

[0140] There are two ways to divide audio segments in a target audio file: direct trimming; and trimming only the vocal segments. A vocal segment refers to an audio segment containing human voices, or an audio segment with lyrics. A vocal segment can also be called a "segment with vocals".

[0141] Optionally, the duration of each audio segment in the target audio is the same, or the duration of each audio segment in the target audio is different.

[0142] For example, such as Figure 8As shown, for the target audio, it is directly cut into two audio segments. The first audio segment is from second 0 to second 3 of the target audio, and the second audio segment is from second 3 to second 7 of the target audio. That is, the start point 1 of the first audio segment is located at second 0 of the target audio, and the end point 1 of the first audio segment is located at second 3 of the target audio. The start point 2 of the second audio segment is also located at second 3 of the target audio, and the end point 2 of the second audio segment is located at second 7 of the target audio. Or, as... Figure 9 As shown, when trimming audio segments, only the human voice segments are trimmed for the target audio. That is, the first audio segment is from second 0 to second 2 of the target audio, the second audio segment is from second 3 to second 5 of the target audio, and the human voice segments from second 2 to third and second 5 to seventh of the target audio will be discarded. That is, the start point 1 corresponding to the first audio segment is located at second 0 of the target audio, the end point 1 corresponding to the first audio segment is located at second 2 of the target audio, the start point 2 corresponding to the second audio segment is located at second 3 of the target audio, and the end point 2 corresponding to the second audio segment is located at second 5 of the target audio.

[0143] It should be noted that the above-mentioned vocal and non-vocal segments are for illustrative purposes only. In actual implementation, vocal segments may contain non-vocal parts. For example, in a 5-second vocal segment, the time containing vocals may be 4 seconds. That is, a vocal segment can be understood as an audio segment where the proportion of vocals is greater than a vocal proportion threshold. The vocal proportion threshold is determined based on expert experience, or it may be set by the developer. In one possible implementation, vocal segments are trimmed and divided by the developer, user, or audio processor based on listening experience, requirements, etc. That is, the division of audio segments is an uncontrollable or unquantifiable process, determined by the preferences of the developer, user, or audio processor. This application embodiment does not limit this. Optionally, non-vocal segments include the intro, interlude, and outro of the target audio, which generally do not have lyrics.

[0144] Optionally, when using the above-mentioned "direct trimming" method of dividing audio segments, the target audio can be preprocessed, including removing at least one of the intro, interlude and outro of the target audio.

[0145] In some embodiments, the j-th audio segment and the (j+1)-th audio segment are audio segments with continuous human voices; or, the j-th audio segment and the (j+1)-th audio segment are audio segments with continuous music.

[0146] Optionally, when the audio segments are divided in the manner described above as "direct cropping", the j-th audio segment and the (j+1)-th audio segment are audio segments with continuous music. That is, the audio segments corresponding to adjacent live photos in at least two live photos are audio segments with continuous music. In other words, the end point corresponding to the j-th audio segment coincides with or is at the same moment as the start point corresponding to the (j+1)-th audio segment.

[0147] Optionally, when the audio segments are divided in the manner described above as "only vocal segments are cropped," the j-th audio segment and the (j+1)-th audio segment are continuous vocal segments, or in other words, the j-th audio segment and the (j+1)-th audio segment are continuous vocal segments. That is, the audio segments corresponding to adjacent live photos in at least two live photos are continuous vocal segments. Alternatively, the lyrics of the audio segments corresponding to adjacent live photos in at least two live photos are continuous. For example, such as... Figure 10 As shown, taking at least two edited live photos as an example of nine live photos, the correspondence between them and the lyrics in the target audio is as follows: the first edited live photo corresponds to the first line of lyrics in the target audio; the second edited live photo corresponds to the second line of lyrics in the target audio; the third edited live photo corresponds to the third line of lyrics in the target audio; the fourth edited live photo corresponds to the fourth line of lyrics in the target audio; the fifth edited live photo corresponds to the fifth line of lyrics in the target audio; the sixth edited live photo corresponds to the sixth line of lyrics in the target audio; the seventh edited live photo corresponds to the seventh line of lyrics in the target audio; the eighth edited live photo corresponds to the eighth line of lyrics in the target audio; and the ninth edited live photo corresponds to the ninth line of lyrics in the target audio.

[0148] In summary, the method provided in this application demonstrates the association between each of at least two live photos and different audio segments in the target audio. That is, adjacent live photos in the edited at least two live photos have corresponding connected audio segments. This ensures that after the edited at least two live photos are published, if a user triggers playback based on the order of the edited at least two live photos, the music played during playback will be continuous and belong to the same target audio. In other words, at least two live photos are linked together by a single target audio, giving the edited at least two live photos a similar musical style. This enhances the association between the edited at least two live photos, facilitates highlighting their common theme, and improves the artistry of the live photo editing process. Furthermore, for ordinary users (such as novice multimedia editors), it allows for simple editing of live photos with clear themes and prominent subjects, reducing the difficulty of live photo editing.

[0149] Furthermore, continuous audio segments can refer to either continuous musical segments or continuous vocal segments. When the continuous audio segments are musical segments, the final edited live photos will have better playback continuity. However, since live photos are generally short (usually 3 seconds), even 9 live photos only require 27 seconds of audio, while the target audio, if it's a song, is approximately 3 to 4 minutes long. Therefore, directly using continuous musical audio might result in the edited live photos playing only the intro to the target audio. Therefore, using continuous vocal segments is preferable; this involves removing non-vocal segments (such as intros, interludes, and outros) during the audio segment editing process, ensuring each live photo corresponds to at least one line of lyrics. This is because, compared to the intro, interlude, and outro—sections without vocals—the average person is more familiar with the lyrics, and the words are more likely to resonate with users, thus creating the atmosphere of at least two live photos, highlighting the theme of at least two live photos, and thereby improving the artistry and effect of live photo editing.

[0150] • How to match live photos with audio clips.

[0151] Optionally, in the above example, the audio segment is matched based on the selection order of at least two live photos selected by the user. That is, the matching of the audio segment is only related to the selection order of at least two live photos. However, the fit between the edited at least two live photos and their matched audio segments may not be high. Therefore, the following method can be used to match live photos and audio segments.

[0152] In some embodiments, the edited at least two live photos have a first order, which is different from the selection order of the at least two live photos; the first order is related to the audio segments corresponding to each of the at least two live photos.

[0153] Optionally, the order of the edited at least two live photos is based on their corresponding audio segments; or, the order of the edited at least two live photos is related to their corresponding audio segments; or, the order of the edited at least two live photos is determined according to their corresponding audio segments.

[0154] In some embodiments, the first order satisfies the order of the audio segments corresponding to the live photos in the target audio; or, the first order is the order of the audio segments corresponding to the live photos in the target audio.

[0155] That is, the above-mentioned process of adding different audio segments from the target audio to each of the at least two live photos to obtain at least two edited live photos includes: matching different audio segments to each of the at least two live photos; adding the audio segment matched by the live photo to each of the at least two live photos; and determining the order of the at least two live photos with added audio segments based on the order of the audio segments matched by each of the at least two live photos in the target audio to obtain at least two edited live photos.

[0156] For example, at least two live photos include four live photos: live photo a1, live photo a2, live photo a3, and live photo a4; the target audio is divided into four audio segments: audio segment 1, audio segment 2, audio segment 3, and audio segment 4. The order of the four audio segments in the target audio is audio segment 1 → audio segment 2 → audio segment 3 → audio segment 4. First, the live photos are matched with the audio segments. The matching results are: live photo a1 matches audio segment 2, live photo a2 matches audio segment 4, live photo a3 matches audio segment 3, and live photo 4 matches audio segment 1. Add a matching audio clip to each of at least two live photos. For example, add audio clip 2 to live photo a1 to get live photo b1, add audio clip 4 to live photo a2 to get live photo b2, add audio clip 3 to live photo a3 to get live photo b3, and add audio clip 1 to live photo a4 to get live photo b4. Reorder these four live photos with added audio clips, based on the order determined by their matching audio clips, to obtain at least two edited live photos: live photo b4, live photo b1, live photo b3, and live photo b2.

[0157] It should be noted that the "first order" of the aforementioned edited at least two live photos refers to the order in which the edited at least two live photos are displayed in the multimedia processing client, or the preview order of the edited at least two live photos, or the order of the edited at least two live photos in the editing interface, or the order of the edited at least two live photos in the preview interface, that is, the arrangement order of the edited at least two live photos in the information flow preview area or information flow interface. However, users may publish the edited at least two live photos without following the generated order, or preview photos without following the generated order, etc., and this application embodiment does not limit this.

[0158] Optionally, step 220 above can be implemented as follows: in response to a selection operation for the target audio in the editing interface, the target audio is determined; based on the degree of matching between each audio segment in the target audio and each of the at least two live photos, the audio segments matching each live photo are determined; and the audio segments matching each live photo are added to each of the at least two live photos to obtain at least two edited live photos.

[0159] Optionally, the target audio includes multiple audio segments. These multiple audio segments are pre-trimmed by the developer; or, the multiple audio segments are trimmed by the server according to a set trimming method; or, the multiple audio segments are trimmed by the user; or, the multiple audio segments are trimmed by the terminal according to a set trimming method; or, the multiple audio segments are trimmed by the multimedia processing client according to a set trimming method.

[0160] Optionally, "determining the audio segment that matches each live photo based on the matching degree between each audio segment in the target audio and each live photo in at least two live photos" includes: determining the audio segment with the highest matching degree with each live photo in at least two live photos based on the matching degree between each audio segment in the target audio and each live photo in at least two live photos; and taking the audio segment with the highest matching degree with the i-th live photo as the audio segment that matches the i-th live photo.

[0161] Optionally, if the number of audio segments included in the target audio is greater than the number of live photos included in the at least two live photos, a first number of consecutive audio segments from the target audio are selected, where the first number is the number of live photos included in the at least two live photos; or, if the number of audio segments included in the target audio is greater than the number of live photos included in the at least two live photos, a prompt message is displayed indicating that adding the target audio to the at least two live photos is not supported. For example, if the target audio includes 4 audio segments and the at least two live photos include 3 live photos, the matching degree of the first 3 audio segments from the target audio and the at least two live photos is calculated respectively.

[0162] For example, at least two live photos include three live photos, namely live photo a1, live photo a2, and live photo a3, and the target audio includes three audio segments, namely audio segment 1, audio segment 2, and audio segment 3. First, the matching degree between each live photo in the at least two live photos and each audio segment in the target audio is calculated. For example, the matching degree between live photo a1 and audio segment 1 is 0.5, the matching degree between live photo a1 and audio segment 2 is 0.8, the matching degree between live photo a1 and audio segment 3 is 0.6, the matching degree between live photo a2 and audio segment 1 is 0.8, the matching degree between live photo a2 and audio segment 2 is 0.6, the matching degree between live photo a2 and audio segment 3 is 0.6, the matching degree between live photo a3 and audio segment 1 is 0.5, the matching degree between live photo a3 and audio segment 2 is 0.9, and the matching degree between live photo a3 and audio segment 3 is 0.8. The audio segment with the highest matching degree with live photo a1 is audio segment 2, the audio segment with the highest matching degree with live photo a2 is audio segment 1, and the audio segment with the highest matching degree with live photo a3 is also audio segment 2. Since the audio segment with the highest matching degree for both live photo a1 and live photo a3 is audio segment 2, audio segment 2 can be added to both live photo a1 and live photo a2; or, compare the matching degree (0.8) between live photo a1 and audio segment 2 and the matching degree (0.9) between live photo a3 and audio segment 2. Since the matching degree between live photo a3 and audio segment 2 is higher than that between live photo a1 and audio segment 2, it is determined that live photo a3 matches audio segment 2; for live photo a1, query the second highest matching degree audio segment, i.e., audio segment 3, and determine whether audio segment 3 has a matching live photo. Since audio segment 3 does not yet exist... Therefore, it is determined that live photo a1 matches audio segment 3; or, when determining the audio segment matched by the live photo, the judgment is made according to the selection order, that is, first judge the live photo a1, and determine that the live photo a1 matches the audio segment 2 with the highest matching degree, then judge the live photo a2, and determine that the live photo a2 matches the audio segment 1 with the highest matching degree, and finally judge the live photo a3. Since the audio segment 2 with the highest matching degree with the live photo a3 has already matched the live photo 1, the second highest matching degree audio segment, i.e., audio segment 3, is judged. Since there is no live photo matching the audio segment 3, it is determined that the live photo a3 matches the audio segment 3.

[0163] In some embodiments, the above-mentioned "determining the audio segments matched by each live photo based on the matching degree between each audio segment in the target audio and each live photo in at least two live photos" includes: for the i-th live photo in at least two live photos, calculating the matching degree between the i-th live photo and the lyrics information of each audio segment; determining the i-th first matching information based on the fact that the matching degree between the i-th live photo in at least two live photos and the lyrics information of the first audio segment in each audio segment is higher than a first preset threshold, wherein the i-th first matching information is used to indicate that the i-th live photo matches the lyrics information of the first audio segment; and determining the audio segments matched by each live photo based on at least two first matching information.

[0164] Optionally, the terminal, server, or multimedia processing client stores a first model, which is used to predict the degree of matching between the lyrics information of the live photo and the audio clip.

[0165] For example, the training data for the first model consists of live photos and lyrics (such as a single lyric). The training dataset is obtained by manually labeling whether the live photo matches the lyrics. The input to the first model is the live photo and lyrics, and the output is the degree of matching between the live photo and the lyrics. During the training of the first model, the training level is evaluated using a loss function. Training of the first model ends when the loss function converges. The first model can be trained using model architectures such as CLIP (Contrastive Language-Image Pre-Training), GAN (Generative Adversarial Network), and CNN (Convolutional Neural Network).

[0166] For example, the first model is used to match whether the items or creatures in the lyrics appear on the cover and / or in the moving image of the live photo. For instance, if there is a tree in the live photo, the live photo will match the lyrics containing the word "tree" or related words more closely than lyrics that do not contain that word.

[0167] In some embodiments, the above-mentioned "determining the audio segments matched by each live photo based on the matching degree between each audio segment in the target audio and each live photo in at least two live photos" includes: for the i-th live photo in at least two live photos, calculating the style matching degree between the i-th live photo and each audio segment in the target audio; determining the i-th second matching information based on the fact that the style matching degree between the i-th live photo in at least two live photos and the second audio segment in each audio segment is higher than a second preset threshold, the i-th second matching information being used to indicate the style matching between the i-th live photo and the second audio segment; and determining the audio segments matched by each live photo based on at least two second matching information.

[0168] Optionally, the terminal, server, or multimedia processing client stores a second model that is used to predict the degree of style matching between live photos and audio clips.

[0169] For example, the training data for the second model consists of live photos and audio clips. The training dataset is obtained by manually labeling whether the live photos and audio clips match. The input to the second model is the live photo and audio clip, and the output is the degree of matching between the live photo and the audio clip. During the training process of the second model, the training progress is evaluated using a loss function. When the loss function converges, the training of the second model ends. The second model can be trained using machine learning methods.

[0170] In summary, the method provided in this application demonstrates that the order of at least two edited live photos is related to their corresponding audio segments. That is, the order of the at least two edited live photos satisfies the order of their corresponding audio segments in the target audio, thus ensuring that the at least two edited live photos are linked together through the target audio order, thereby improving the correlation between the at least two edited live photos.

[0171] Furthermore, the method for matching at least two live photos with audio clips is demonstrated. Besides determining the corresponding audio clip based on the selection order of the live photos, as mentioned above, matching can also be based on the degree of matching between the live photos and audio clips. This ensures that the live photos and audio clips are not completely unrelated, strengthening the connection between them and improving the matching degree between the edited at least two live photos and audio clips, thus enhancing the thematic and artistic quality of the edited at least two live photos. Moreover, it eliminates the need for users to adjust the order of at least two live photos based on the audio clips after selecting the target audio, improving the editing efficiency of at least two live photos.

[0172] Furthermore, two methods for matching live photos with audio clips are demonstrated: matching based on lyrics information or the style of the audio clips. Both methods can calculate the degree of match between the live photos and audio clips that aligns with the user's aesthetic preferences and needs. This eliminates the need for users to manually adjust the order of live photos by listening to each line of the audio clip to achieve a better match, thus improving the editing efficiency of at least two live photos.

[0173] • Set the speed change points.

[0174] In some embodiments, the method further includes: adding a beat-and-speed-change point to each of at least two live photos, wherein the beat-and-speed-change point is the same as the audio speed-change point of the audio segment corresponding to the live photo, the beat-and-speed-change point is used to change the playback speed of the live photo, and the audio speed-change point is a transition point of at least one of frequency, speed, pitch, volume, and rhythm in the audio segment.

[0175] Optionally, the timing and speed adjustment points of the live photo can be set according to the audio segment corresponding to the live photo; or, the speed adjustment curve of the live photo can be updated according to the audio segment corresponding to the live photo; or, the playback speed of the live photo can be updated according to the audio segment corresponding to the live photo.

[0176] Optionally, the audio tempo change point of an audio segment is a transition point of at least one of the following: frequency, tempo, pitch, volume, and rhythm. Frequency can be determined by sampling and performing a Fourier transform on the audio segment to obtain its frequency variation trend. The audio tempo change point determined based on frequency is the time point (moment) when the frequency changes, i.e., the time point from high frequency to low frequency, or from low frequency to high frequency. Tempo refers to the number of beats (pulses) within a fixed time period; for example, the tempo of an audio segment can refer to the number of beats per minute. Pitch indicates the frequency of the human voice in the audio segment. Volume indicates the volume of the human voice in the audio segment. Rhythm refers to the time interval between notes, i.e., the variation in the duration of notes; rhythm is closely related to tempo.

[0177] For example, the point in time when an audio segment transitions from a fast tempo to a slow tempo is an audio tempo change point, and similarly, the point in time when a slow tempo transitions to a fast tempo is also an audio tempo change point. Alternatively, if the degree of tempo change within a fixed duration of an audio segment exceeds a certain threshold, then there exists an audio tempo change point within that fixed duration. This tempo change point can be the middle, beginning, end, or other time within that fixed duration.

[0178] Optionally, the audio speed adjustment point of an audio segment is set by the developer, user, or audio processor; or, the audio speed adjustment point of an audio segment is obtained by the terminal, server, or multimedia processing client according to the set program.

[0179] For example, such as Figure 11 As shown, the tempo change point (i.e., the audio tempo change point) corresponding to the rhythm curve 60 of the audio clip is point 61, which corresponds to 70% of the audio clip's playback time. If the audio clip's duration is 3 seconds, then the corresponding audio tempo change point is at 3 × 70% = 2.1 seconds. For the live photo matching this audio clip, a timing tempo change point should be set for the live photo. This timing tempo change point coincides with the audio clip's tempo change point. The tempo change curve of the live photo after setting the timing tempo change point is as follows: Figure 11 As shown in curve 62, the speed change curve 62 is used to indicate the change in playback speed of the live photos. For Figure 11 The speed change curve 62 shown in the figure plays the live photo at the original speed (i.e., 1x speed) from 0 to approximately 2.1 seconds. Around 2.1 seconds, the playback speed of the live photo decreases over time until it reaches 0.1x speed. This change in speed from 1x to 0.1x is set by the developer or the user. For example, the speed change curve from 1x to 0.1x satisfies a function with the audio speed change point as the origin, time as the x-axis, and the speed change as the y-axis.

[0180] Optionally, after setting the speed adjustment point, the playback duration of the live photo will change. If the playback duration of the live photo is less than the playback duration of the corresponding audio segment, the playback speed of the live photo will be reduced, such as updating the playback speed of the live photo to 0.8x. For example, such as... Figure 12 As shown in part (1), the initial playback duration of the live photo is 2 seconds. After setting the beat and slowing down, the playback duration becomes 2.4 seconds, but the playback duration of the audio clip is 3 seconds. Therefore, as shown in part (1), the initial playback duration of the live photo is 2 seconds. After setting the beat and slowing down, the playback duration becomes 2.4 seconds. However, the playback duration of the audio clip is 3 seconds. Figure 12 The playback speed of the updated live photos shown in part (2) is 0.8x, which makes the playback time of the updated live photos 2.4 / 0.8 = 3 seconds.

[0181] Optionally, if the playback duration of a live photo is longer than the playback duration of the corresponding audio segment, the playback speed of the live photo can be increased, such as updating the playback speed of the live photo to 1.2x; or, as... Figure 13 As shown, when the playback duration of a live photo is longer than the playback duration of the audio segment corresponding to that live photo, the live photo is cropped, that is, the playback duration of the live photo is cropped to be equal to the playback duration of the audio segment corresponding to that live photo, and the part that exceeds the end point of the audio segment is cropped and removed.

[0182] In summary, the method provided in this application automatically adds beat-based speed-changing points to live photos based on the audio speed-changing points of the audio clip itself. This allows the playback speed of the live photos to change according to at least one of the rhythm, pitch, speed, frequency, and volume of the audio clip, thereby enhancing the artistry of the live photos during playback. Furthermore, it eliminates the need for users to manually set beat-based speed-changing points to achieve this effect, thus improving the editing efficiency of at least two live photos.

[0183] In some embodiments, the method further includes: in response to a selection operation for at least two live photos, displaying recommended audio information based on the at least two live photos, the recommended audio information indicating audio that matches at least one of the content and style of the at least two live photos.

[0184] Optionally, the terminal, server, or multimedia processing client stores a third model, which is used to predict the live photo and the matching target audio.

[0185] For example, the training data for the third model consists of live photos and audio. The training dataset is obtained by manually labeling whether the live photo matches the audio. The input to the third model is the live photo, and the output is the audio that matches the live photo. During the training process of the third model, the training level is evaluated using a loss function. When the loss function converges, the training of the third model ends. The third model can be trained using machine learning methods.

[0186] It should be noted that the above-mentioned "setting up a feed preview area," "publishing to a social interaction platform," "saving to an album," "setting the association between live photos and audio clips," "matching method for live photos and audio clips," and "setting beat / speed adjustment points" can be implemented as independent embodiments or as combined embodiments. For example, the above-mentioned "setting up a feed preview area" and "publishing to a social interaction platform" can be implemented in combination, that is, while setting up a feed preview area in the editing interface, the user can also publish at least two edited live photos to the feed interface corresponding to the social interaction platform by publishing the edited at least two live photos. Optionally, when the boundary interface supports displaying feed preview areas corresponding to different social interaction platforms, the edited at least two live photos are published to the feed interface of the social interaction platform corresponding to the currently displayed feed preview area. "Setting up a feed preview area" and "setting the association between live photos and audio clips" can be implemented in combination, that is, for the display order of at least two live photos displayed in the feed preview area, the audio clips corresponding to adjacent live photos are set to be consecutive. The options "Setting the feed preview area" and "Matching method for live photos and audio clips" can be implemented in combination. This means that for at least two live photos displayed in the feed preview area, their corresponding audio clips are matched using the method shown in "Matching method for live photos and audio clips." The options "Setting the feed preview area" and "Setting beat / speed change points" can also be implemented in combination. This means that for at least two edited live photos displayed in the feed preview area, beat / speed change points are set according to the method shown in "Setting beat / speed change points." The options "Publishing to social media platforms" and "Setting the association between live photos and audio clips" can also be implemented in combination. This means that the relationship between at least two edited live photos and audio clips published to social media platforms conforms to the association relationship shown in "Setting the association between live photos and audio clips." The options "Publishing to social media platforms" and "Matching method for live photos and audio clips" mean that the audio clips matched with at least two edited live photos published to social media platforms are matched based on the method shown in "Matching method for live photos and audio clips." "Publish to social media platforms" and "Set beat / speed-up points" can be implemented in combination. That is, at least two edited live photos published to social media platforms are set with the beat / speed-up points shown in "Set beat / speed-up points". "Set the association between live photos and audio clips" and "Set the matching method between live photos and audio clips" can be implemented in combination. That is, firstly, the matching method shown in "Set the matching method between live photos and audio clips" is used to match audio clips for at least two live photos. Then, according to the order of the audio clips in the target audio (i.e., the method shown in "Set the association between live photos and audio clips"), matching audio clips are added to at least two live photos to obtain at least two edited live photos.The functions "Setting the association between live photos and audio clips" and "Setting beat / speed change points" can be implemented in combination. This means that at least two edited live photos that satisfy the association relationship shown in "Setting the association between live photos and audio clips" also have the beat / speed change points shown in "Setting beat / speed change points." The functions "Matching method for live photos and audio clips" and "Setting beat / speed change points" can be implemented in combination. This means that at least two edited live photos matched using the "Matching method for live photos and audio clips" also have the beat / speed change points shown in "Setting beat / speed change points." The functions "Setting the feed preview area" and "Saving to album" can be implemented in combination. This means that at least two live photos displayed in the feed preview area can be saved to the album. The functions "Publishing to social media platforms" and "Saving to album" can be implemented in combination. This means that at least two edited live photos can be saved to the album first and then published to a social media platform. The "Association of Live Photos and Audio Clips" and "Save to Album" functions can be combined, meaning that in at least two edited Live Photos saved to the album, the audio clips corresponding to adjacent Live Photos must be consecutive. The "Matching Method of Live Photos and Audio Clips" and "Save to Album" functions can also be combined, meaning that the matching relationship of at least two edited Live Photos saved to the album must satisfy the matching relationship shown in "Matching Method of Live Photos and Audio Clips". The "Setting Beat Speed ​​Change Points" and "Save to Album" functions can also be combined, meaning that at least two edited Live Photos saved to the album must have the beat speed change points shown in "Setting Beat Speed ​​Change Points". In addition, combinations of 3x3, 4x4, 5x5, or even 6x6 can be used. For example, "Setting up a feed preview area," "Setting the association between live photos and audio clips," and "Setting the matching method for live photos and audio clips" can be implemented in combination. For the display order of at least two live photos displayed in the feed preview area, the audio clips corresponding to adjacent live photos are set to be consecutive, and their corresponding audio clips are matched using the method shown in "Setting the matching method for live photos and audio clips." "Setting up a feed preview area," "Publishing to a social interaction platform," and "Setting the association between live photos and audio clips" can be implemented in combination. "Setting up a feed preview area," "Publishing to a social interaction platform," and "Setting the matching method for live photos and audio clips" can be implemented in combination. "Setting up a feed preview area," "Publishing to a social interaction platform," and "Setting beat / speed-up points" can be implemented in combination. "Setting up a feed preview area," "Setting the association between live photos and audio clips," and "Setting beat / speed-up points" can be implemented in combination. "Setting up a feed preview area," "Setting the matching method for live photos and audio clips," and "Setting beat / speed-up points" can be implemented in combination. "Publish to social media platforms", "Set up associations between live photos and audio clips", and "Matching methods for live photos and audio clips" can be implemented in combination.The functions of "posting to social media platforms," ​​"matching live photos and audio clips," and "setting beat / speed adjustment points" can be implemented in combination. (This is repeated four times in the original text.) The functions "Setting up a preview area for the feed," "Publishing to a social media platform," "Setting the association between live photos and audio clips," and "Setting beat / speed adjustment points" can be implemented in combination. The functions "Setting up a preview area for the feed," "Publishing to a social media platform," "Setting the association between live photos and audio clips," "Matching methods for live photos and audio clips," and "Setting beat / speed adjustment points" can be implemented in combination. The functions "Setting up a preview area for the feed," "Publishing to a social media platform," and "Save to album" can be implemented in combination. The functions "Setting up a preview area for the feed," "Save to album," and "Setting the association between live photos and audio clips" can be implemented in combination. The functions "Setting up a preview area for the feed," "Save to album," and "Matching methods for live photos and audio clips" can be implemented in combination. The functions "Setting up a preview area for the feed," "Save to album," and "Setting beat / speed adjustment points" can be implemented in combination. The functions "Publishing to a social media platform," "Save to album," and "Setting the association between live photos and audio clips" can be implemented in combination. The functions "Publishing to a social media platform," "Save to album," and "Matching methods for live photos and audio clips" can be implemented in combination. The functions "Publish to social media platforms," ​​"Save to album," and "Set beat / speed adjustment points" can be implemented in combination. The functions "Save to album," "Set the association between live photos and audio clips," and "Set the matching method for live photos and audio clips" can be implemented in combination. The functions "Set up a preview area in the feed," "Publish to social media platforms," ​​"Save to album," and "Set the association between live photos and audio clips" can be implemented in combination."Setting up a preview area for the information stream," "publishing to a social interaction platform," "saving to the album," and "matching live photos with audio clips" can be implemented in combination. It should be noted that the embodiments of this application only list some of the combinations of the above solutions, but the protection scope of the embodiments of this application is not limited thereto.

[0187] Figure 14 This diagram illustrates a structural block diagram of a live photo editing apparatus provided in an exemplary embodiment of this application. The apparatus has the functionality to implement the above-described example of a live photo editing method; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a terminal or multimedia processing client as described above, or it can be located within a terminal or multimedia processing client. The apparatus includes: a display module 410, an editing module 420, and a preview module 430.

[0188] Display module 410 is configured to display an editing interface for the at least two live photos in response to a selection operation for at least two live photos, the editing interface including a selection area for at least one candidate audio.

[0189] Editing module 420 is configured to, in response to a selection operation for a target audio in the editing interface, add different audio segments of the target audio to the at least two live photos respectively to obtain at least two edited live photos, wherein the target audio is one of the at least one candidate audio;

[0190] The preview module 430 is configured to display a preview of the first live photo in response to a preview operation on the first live photo of the at least two edited live photos.

[0191] In some embodiments, the editing module 420 is further configured to, in response to a selection operation for the target audio in the editing interface, add the j-th audio segment of the target audio to the i-th live photo among the at least two live photos; and add the j+1-th audio segment of the target audio to the (i+1)-th live photo among the at least two live photos;

[0192] Where i and j are both positive integers; the j-th audio segment is consecutive to the (j+1)-th audio segment.

[0193] In some embodiments, the j-th audio segment and the (j+1)-th audio segment are audio segments with continuous human voices; or, the j-th audio segment and the (j+1)-th audio segment are audio segments with continuous music.

[0194] In some embodiments, the display module 410 is further configured to display an information flow preview area on the editing interface, the information flow preview area being used to indicate the preview effect of at least two edited live photos after they are published to the information flow interface.

[0195] In some embodiments, the display module 410 includes a first playback submodule.

[0196] The first playback submodule is configured to play the first live photo in response to a trigger operation in the information stream preview area for the first live photo among the at least two edited live photos, wherein the audio of the first live photo includes an audio segment corresponding to the first live photo.

[0197] In some embodiments, the display module 410 further includes a second playback submodule.

[0198] The second playback submodule is used to play the second live photo in response to a trigger operation on the second live photo. The second live photo is a live photo adjacent to the first live photo. The audio of the second live photo includes an audio segment corresponding to the second live photo. The audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo.

[0199] In some embodiments, the edited at least two live photos have a first order, which is different from the selection order of the at least two live photos; the first order is related to the audio segments corresponding to each of the at least two live photos.

[0200] In some embodiments, the first order satisfies the order of the audio segments corresponding to the live photo in the target audio.

[0201] In some embodiments, the editing module 420 includes a determining submodule, a matching submodule, and an adding submodule.

[0202] A determination submodule is used to determine the target audio in response to a selection operation for the target audio in the editing interface;

[0203] The matching submodule is used to determine the audio segments that each live photo matches based on the degree of matching between each audio segment in the target audio and each live photo in the at least two live photos.

[0204] A submodule is added to add audio segments that match each of the at least two live photos to obtain edited at least two live photos.

[0205] In some embodiments, the matching submodule is further configured to: calculate the degree of matching between the i-th live photo and the lyrics information of each audio segment for the i-th live photo among the at least two live photos; determine the i-th first matching information based on the fact that the degree of matching between the i-th live photo and the lyrics information of the first audio segment among the at least two live photos is higher than a first preset threshold, wherein the i-th first matching information is used to indicate that the i-th live photo matches the lyrics information of the first audio segment; and determine the audio segments matched by each live photo based on at least two sets of the first matching information.

[0206] In some embodiments, the editing module 420 is further configured to add a beat-and-speed-change point to each of the at least two live photos, wherein the beat-and-speed-change point is the same as the audio speed-change point of the audio segment corresponding to the live photo, the beat-and-speed-change point is used to change the playback speed of the live photo, and the audio speed-change point is a transition point of at least one of frequency, speed, pitch, volume, and rhythm in the audio segment.

[0207] In some embodiments, the apparatus further includes a save module and / or a publish module.

[0208] A save module is configured to export the edited at least two Live Photos to an album in response to an export operation on the edited at least two Live Photos.

[0209] The publishing module is used to display the edited at least two live photos in the information flow interface in response to the publishing operation for the edited at least two live photos.

[0210] In some embodiments, the device further includes a viewing module.

[0211] The viewing module is configured to, in response to a trigger operation on the information flow interface for a first live photo among the at least two edited live photos, play the first live photo, the audio of which includes an audio segment corresponding to the first live photo; and, in response to a trigger operation on a second live photo or after the first live photo has finished playing, play the second live photo, which is the next live photo adjacent to the first live photo, the audio of which includes an audio segment corresponding to the second live photo, and the audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo.

[0212] It should be noted that the specific limitations of the embodiments of the one or more live photo editing devices provided above can be found in the limitations of the live photo editing method above, and will not be repeated here. Each module of the above device can be implemented entirely or partially by software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0213] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0214] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one computer program, which is loaded and executed by the processor to implement the live photo editing method provided in the above-described method embodiments.

[0215] Figure 15 This illustration shows a schematic diagram of a computer device provided in an exemplary embodiment of this application. The computer device 500 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), or MP4 player (Moving Picture Experts Group Audio Layer IV). The computer device 500 may also be referred to as a user device, portable terminal, or other names. Typically, the computer device 500 includes a processor 501 and a memory 502.

[0216] Processor 501 may include one or more processing cores, such as a quad-core processor, a penta-core processor, etc. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0217] The memory 502 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 502 is used to store at least one instruction, which is executed by the processor 501 to implement the gift-giving method based on a live-streaming room provided in this embodiment.

[0218] In some embodiments, the computer device 500 may also optionally include a peripheral device interface 503 and at least one peripheral device. The processor 501, memory 502, and peripheral device interface 503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 504, a touch display screen 505, a camera assembly 506, an audio circuit 507, and a power supply 508.

[0219] Peripheral device interface 503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 501 and memory 502. In some embodiments, processor 501, memory 502 and peripheral device interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 501, memory 502 and peripheral device interface 503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0220] The radio frequency (RF) circuit 504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 504 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or Wi-Fi (Wireless Fidelity) networks. In some embodiments, the RF circuit 504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0221] The touch display screen 505 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. The touch display screen 505 also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to the processor 501 for processing. The touch display screen 505 is used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one touch display screen 505, located on the front panel of the computer device 500; in other embodiments, there may be at least two touch display screens, respectively located on different surfaces of the computer device 500 or in a folded design; in some embodiments, the touch display screen 505 may be a flexible display screen, located on a curved or folded surface of the computer device 500. Furthermore, the touch display screen 505 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The touch display screen 505 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0222] Camera assembly 506 is used to capture images or videos. Optionally, camera assembly 506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is used for video calls or selfies, and the rear-facing camera is used for taking photos or videos. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, and a wide-angle camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, and panoramic shooting and VR shooting by fusion of the main camera and the wide-angle camera. In some embodiments, camera assembly 506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0223] Audio circuitry 507 provides an audio interface between the user and computer device 500. Audio circuitry 507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to processor 501 for processing, or input to radio frequency circuitry 504 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different location within computer device 500. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from processor 501 or radio frequency circuitry 504 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, audio circuitry 507 may also include a headphone jack.

[0224] Power supply 508 is used to supply power to the various components in computer device 500. Power supply 508 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 508 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0225] In some embodiments, the computer device 500 further includes one or more sensors 509. The one or more sensors 509 include, but are not limited to, an accelerometer 510, a gyroscope 511, a pressure sensor 512, an optical sensor 513, and a proximity sensor 514.

[0226] Accelerometer 510 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by computer device 500. For example, accelerometer 510 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 501 can control touch screen 505 to display the user interface in landscape or portrait view based on the gravitational acceleration signal collected by accelerometer 510. Accelerometer 510 can also be used for games or to collect user motion data. Gyroscope sensor 511 can detect the orientation and rotation angle of computer device 500. Gyroscope sensor 511 can work in conjunction with accelerometer 510 to collect 3D movements of the user on computer device 500. Based on the data collected by gyroscope sensor 511, processor 501 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0227] The pressure sensor 512 can be disposed on the side bezel of the computer device 500 and / or on the lower layer of the touch display screen 505. When the pressure sensor 512 is disposed on the side bezel of the computer device 500, it can detect the user's grip signal on the computer device 500 and perform left / right hand recognition or quick operation based on the grip signal. When the pressure sensor 512 is disposed on the lower layer of the touch display screen 505, it can control the operable controls on the UI interface based on the user's pressure operation on the touch display screen 505. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0228] An optical sensor 513 is used to collect ambient light intensity. In one embodiment, the processor 501 can control the display brightness of the touch screen 505 based on the ambient light intensity collected by the optical sensor 513. Specifically, when the ambient light intensity is high, the display brightness of the touch screen 505 is increased; when the ambient light intensity is low, the display brightness of the touch screen 505 is decreased. In another embodiment, the processor 501 can also dynamically adjust the shooting parameters of the camera assembly 506 based on the ambient light intensity collected by the optical sensor 513.

[0229] The proximity sensor 514, also known as a distance sensor, is typically located on the front of the computer device 500. The proximity sensor 514 is used to detect the distance between the user and the front of the computer device 500. In one embodiment, when the proximity sensor 514 detects that the distance between the user and the front of the computer device 500 is gradually decreasing, the processor 501 controls the touchscreen display 505 to switch from a screen-on state to a screen-off state; when the proximity sensor 514 detects that the distance between the user and the front of the computer device 500 is gradually increasing, the processor 501 controls the touchscreen display 505 to switch from a screen-off state to a screen-on state.

[0230] Those skilled in the art will understand that the above structure does not constitute a limitation on the computer device 500, and may include more or fewer components than shown in the figure, or combine certain components, or use different component arrangements.

[0231] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the virtual object control method provided in the above-described method embodiments.

[0232] This application also provides a computer program product, which includes at least one computer program stored in a computer-readable storage medium; the at least one computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to execute the virtual object control method provided in the above method embodiments.

[0233] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0234] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0235] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for editing live photos, characterized in that, The method includes: In response to a selection operation for at least two live photos, an editing interface for the at least two live photos is displayed, the editing interface including a selection area for at least one candidate audio. In response to the selection operation for the target audio in the editing interface, different audio segments of the target audio are added to the at least two live photos respectively to obtain at least two edited live photos, wherein the target audio is one of the at least one candidate audio; In response to a preview operation on the first of the at least two edited live photos, a preview of the first live photo is displayed.

2. The method according to claim 1, characterized in that, The response to the selection operation for the target audio in the editing interface, adding different audio segments of the target audio to the at least two live photos to obtain edited at least two live photos, includes: In response to the selection operation for the target audio in the editing interface, the j-th audio segment of the target audio is added to the i-th live photo among the at least two live photos; the (i+1)-th audio segment of the target audio is added to the (i+1)-th live photo among the at least two live photos; Where i and j are both positive integers; the j-th audio segment is consecutive to the (j+1)-th audio segment.

3. The method according to claim 2, characterized in that, The j-th audio segment and the (j+1)-th audio segment are audio segments with continuous human voices; Alternatively, the j-th audio segment and the (j+1)-th audio segment are audio segments that are musically continuous.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The editing interface displays a feed preview area, which is used to indicate the preview effect of at least two edited live photos after they are published to the feed interface.

5. The method according to claim 4, characterized in that, The step of displaying a preview of the first live photo in response to a preview operation of the first live photo among the at least two edited live photos includes: In response to a trigger operation in the information stream preview area for the first of the at least two edited live photos, the first live photo is played, and the audio of the first live photo includes an audio segment corresponding to the first live photo.

6. The method according to claim 5, characterized in that, The method further includes: In response to a trigger operation on a second live photo, the second live photo is played. The second live photo is a live photo adjacent to the first live photo. The audio of the second live photo includes an audio segment corresponding to the second live photo. The audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo.

7. The method according to any one of claims 1 to 6, characterized in that, The edited at least two live photos have a first order, which is different from the selection order of the at least two live photos; The first order is related to the audio segments corresponding to each of the at least two live photos.

8. The method according to claim 7, characterized in that, The first order satisfies the order of the audio segments corresponding to the live photo in the target audio.

9. The method according to any one of claims 1 to 8, characterized in that, The response to the selection operation for the target audio in the editing interface, adding different audio segments of the target audio to the at least two live photos to obtain edited at least two live photos, includes: In response to a selection operation for the target audio in the editing interface, the target audio is determined; Based on the degree of matching between each audio segment in the target audio and each of the at least two live photos, the audio segments that each live photo matches are determined. Add audio segments that match each of the at least two live photos to each of the live photos to obtain at least two edited live photos.

10. The method according to claim 9, characterized in that, The step of determining the audio segment that matches each live photo based on the matching degree between each audio segment in the target audio and each live photo in the at least two live photos includes: For the i-th live photo among the at least two live photos, calculate the degree of matching between the i-th live photo and the lyrics information of each audio segment; Based on the fact that the matching degree between the i-th live photo in the at least two live photos and the lyrics information of the first audio segment in each audio segment is higher than a first preset threshold, the i-th first matching information is determined, and the i-th first matching information is used to indicate that the i-th live photo matches the lyrics information of the first audio segment. Based on at least two of the first matching information, the audio segments that are matched to each of the live photos are determined.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: Add a beat-and-speed-change point to each of the at least two live photos. The beat-and-speed-change point is the same as the audio speed-change point of the audio segment corresponding to the live photo. The beat-and-speed-change point is used to change the playback speed of the live photo. The audio speed-change point is a transition point of at least one of frequency, speed, pitch, volume, and rhythm in the audio segment.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: In response to the export operation for the at least two edited Live Photos, the at least two edited Live Photos are exported to the album; And / or, in response to a publishing operation for at least two edited Live Photos, display at least two edited Live Photos in the feed interface.

13. The method according to claim 12, characterized in that, The method further includes: In response to a trigger operation in the information flow interface for the first live photo among the at least two edited live photos, the first live photo is played, and the audio of the first live photo includes an audio segment corresponding to the first live photo. In response to a trigger operation on the second live photo or after the first live photo has finished playing, the second live photo is played. The second live photo is the next live photo adjacent to the first live photo. The audio of the second live photo includes an audio segment corresponding to the second live photo, and the audio segment corresponding to the second live photo is continuous with the audio segment corresponding to the first live photo.

14. A live photo editing device, characterized in that, The device includes: A display module is configured to display an editing interface for the at least two live photos in response to a selection operation for at least two live photos, the editing interface including a selection area for at least one candidate audio. An editing module is configured to, in response to a selection operation for a target audio in the editing interface, add different audio segments of the target audio to the at least two live photos respectively to obtain at least two edited live photos, wherein the target audio is one of the at least one candidate audio; A preview module is configured to display a preview of the first live photo in response to a preview operation on the first live photo of the at least two edited live photos.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the live photo editing method as described in any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the live photo editing method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the live photo editing method as described in any one of claims 1 to 13.

Images