Method, device, equipment, storage medium and computer program product for setting beautification material

Abstract

This application discloses a method, apparatus, device, and storage medium for setting enhancement materials, relating to the fields of computer and internet technology. The method includes: displaying a first video and a first control for setting the first video as enhancement material in a first interface for audio playback, wherein the first video is an audio-related video; displaying at least two setting options in response to an operation on the first control, wherein different setting options are used to set the first video as enhancement material to be displayed in different scenarios; and setting the first video as enhancement material to be displayed in the target scenario in response to an operation on a target setting option among the at least two setting options. Therefore, this application significantly reduces the operational complexity of setting enhancement materials.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer and Internet technology, and in particular to a method, device, equipment and storage medium for setting beautification materials. Background Art

[0002] Currently, users can set beautifying materials such as incoming call ringtones in their mobile phones.

[0003] In the related art, a user can install an application dedicated to setting beautification materials in a mobile phone, manually import the audio that he wants to set as the incoming call ringtone in the above application, and then perform some necessary setting operations on the imported audio to set the above audio as the incoming call ringtone.

[0004] However, the above method of setting beautification materials has high operational complexity. Summary of the invention

[0005] The embodiment of the present application provides a method, device, equipment and storage medium for setting beautification materials. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for setting a beautification material is provided, the method comprising:

[0007] In a first interface of audio playback, a first video and a first control for setting the first video as a beautification material are displayed, wherein the first video is a video related to the audio;

[0008] In response to an operation on the first control, at least two setting options are displayed, and different setting options are used to set the first video as a beautification material displayed in different scenes;

[0009] In response to an operation on a target setting option among the at least two setting options, the first video is set as a beautification material displayed in a target scene.

[0010] According to one aspect of an embodiment of the present application, a device for setting beautification material is provided, the device comprising:

[0011] An interface display module, used for displaying a first video and a first control for setting the first video as a beautification material in a first interface of audio playback, wherein the first video is a video related to the audio;

[0012] An option display module, used for displaying at least two setting options in response to an operation on the first control, wherein different setting options are used for setting the first video as a beautification material displayed in different scenes;

[0013] The material setting module is used to set the first video as a beautification material displayed in a target scene in response to an operation on a target setting option among the at least two setting options.

[0014] According to one aspect of an embodiment of the present application, a terminal device is provided, the terminal device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned method for setting beautification materials.

[0015] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned method for setting beautification materials.

[0016] According to one aspect of the embodiments of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a terminal device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the terminal device executes the above-mentioned method for setting beautification materials.

[0017] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0018] On the one hand, by setting the video related to the audio displayed during the audio playback as the beautification material, that is, when you encounter a favorite video clip while listening to a song or watching a music video, you can set the currently playing video as the beautification material with one click, without manually importing the audio that needs to be set as the beautification material as in the related art. Therefore, the operational complexity of setting the beautification material is reduced and the setting efficiency of the beautification material is improved.

[0019] On the other hand, when setting a video as a beautification material, multiple setting options are provided, allowing users to choose to set the video as a beautification material displayed in different scenes (such as incoming call scene, charging scene, prompt sound scene, wallpaper scene, ringtone scene, etc.), which fully enhances the flexibility of beautification material setting. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0021] Figure 2 It is a flow chart of a method for setting beautification materials provided by an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a first control provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a first control provided by another embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a first control provided by another embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a first control provided by another embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a first control provided by another embodiment of the present application;

[0027] Figure 8 is a schematic diagram of a first control provided by another embodiment of the present application;

[0028] Fig. 9 is a schematic diagram of setting options for a scenario provided by an embodiment of the present application;

[0029] Fig.10 is a schematic diagram of setting options for a scene provided by another embodiment of the present application;

[0030] Fig.11 This is a schematic diagram of an interface provided by an embodiment of the present application;

[0031] Fig.12 is a flow chart of a method for setting beautification materials provided by another embodiment of the present application;

[0032] Fig.13 is a schematic diagram of a preview effect provided by an embodiment of the present application;

[0033] Fig.14 is a schematic diagram of a preview effect provided by another embodiment of the present application;

[0034] Fig.15 is a schematic diagram of a preview effect provided by another embodiment of the present application;

[0035] Fig.16 is a schematic diagram of a preview effect provided by another embodiment of the present application;

[0036] Fig.17 is a schematic diagram of a preview effect provided by another embodiment of the present application;

[0037] Fig.18 is a flow chart of a method for setting beautification materials provided by another embodiment of the present application;

[0038] Fig.19 is a schematic diagram of a video production interface provided by an embodiment of the present application;

[0039] Fig. 20 is a schematic diagram of a video production interface provided by another embodiment of the present application;

[0040] Fig.21 It is an architecture diagram of an AI (Artificial Intelligence) model provided by an embodiment of the present application;

[0041] Fig. 22 is an architecture diagram of an AI (Artificial Intelligence) model provided in another embodiment of the present application;

[0042] Fig.23 is a schematic diagram of a function switch provided by an embodiment of the present application;

[0043] Fig.24 It is a block diagram of a device for setting beautification materials provided by an embodiment of the present application;

[0044] Fig.25 is a block diagram of a device for setting beautification materials provided by another embodiment of the present application;

[0045] Fig.26 This is a structural block diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0047] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include: a terminal device 11 and a server 12.

[0048] The terminal device 11 may be an electronic device such as a mobile phone, a tablet computer, a PC (Personal Computer), a wearable device, a VR (Virtual Reality) device, an AR (Augmented Reality) device, a vehicle-mounted device, etc., and this application does not limit this. A client running a target application may be installed in the terminal device 11. For example, the target application may be an application for playing audio, a music playing application, a social application, an interactive entertainment application, etc., and this application does not limit this.

[0049] The server 12 may be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The server 12 may be a background server of the target application program, and is used to provide background services for the client of the target application program.

[0050] The terminal device 11 can communicate with the server 12 via a network.

[0051] Please refer to Figure 2 , which shows a flow chart of a method for setting beautification materials provided by an embodiment of the present application. The execution subject of each step of the method can be Figure 1 The terminal device 11 in the implementation environment of the scheme shown, for example, the execution subject of each step can be a client. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "client". The method can include at least one of the following steps (210-230):

[0052] Step 210: In a first interface of audio playback, a first video and a first control for setting the first video as a beautification material are displayed, wherein the first video is a video related to the audio.

[0053] First interface: the interface displayed by the client when playing audio. Optionally, the background music in the first video is the audio. Optionally, when playing audio, the screen content of the first video with the audio as background music is played at the same time. Optionally, a progress bar for the audio playback is displayed in the first interface.

[0054] First video: video related to the audio. Optionally, the background music of the first video is the audio. Optionally, the audio is the original sound or dubbing of the first video. Optionally, the first video is a video generated based on the audio. In some embodiments, the first interface of audio playback is provided by the application that plays the audio. This application does not limit the application that plays the audio. Optionally, the application that plays the audio is an audio playback application, such as a music application. Optionally, the application that plays the audio is a video playback application, such as a short video platform. In addition, the application that plays the audio includes but is not limited to game applications, live broadcast applications, and the like. When the application that plays the audio is a music application, when the audio playback is triggered, the first interface of the audio playback is displayed, and the first video corresponding to the audio is displayed at the same time. Optionally, the audio is at least one of the music databases in the music application. Optionally, there is at least one video related to the audio. Optionally, the video is a video that uses the audio as background music. When the audio is played, in the first interface of the audio playback, the first video and the first control for setting the first video as the beautification material are displayed. The explanation of other applications refers to the music application, which will not be repeated here.

[0055] In some embodiments, the first video is a video in a music video resource library of an application that plays audio. In some embodiments, the application that plays audio stores videos uploaded by users. Optionally, the application that plays audio considers the videos uploaded by users as videos in the music video resource library. Optionally, the same audio corresponds to at least one video. For song a, 100 users created videos for song a respectively, and 100 different videos were obtained. These 100 different videos can all be considered as videos corresponding to song a.

[0056] In some embodiments, the first video is a video generated by an application that plays audio. Optionally, the application that plays audio can generate a video based on the audio without the need for the user to upload it. Optionally, the application that plays audio generates a video based on the rhythm information of the audio. Optionally, the picture change of the video is positively correlated with the change of the rhythm information. For example, in places where the rhythm of the audio is relatively slow, the picture change rate of the video is low, and the picture change area accounts for a low proportion of the total picture area. Relatively speaking, in places where the rhythm of the audio is relatively intense, the picture change rate of the video is high, and the picture change area accounts for a high proportion of the total picture area. Optionally, the application that plays audio generates a video based on keyword information in the audio. Optionally, the application that plays audio extracts text information in the audio, and determines the text whose frequency of occurrence of text information is greater than a first threshold as keyword information. Optionally, a picture or video clip corresponding to the keyword information is extracted from a database based on the keyword information. Optionally, the picture or video clip corresponding to the keyword information is merged to obtain a video corresponding to the audio. In other embodiments, when the audio is a song, a video can be generated based on the lyrics of the song. In some embodiments, a video corresponding to the song is automatically generated based on the lyrics of the song by an AI model. Optionally, how to generate a video using an AI model is described in the following embodiments. Fig.21 This application does not limit the specific way in which the application that plays audio generates video by itself. Please refer to the explanation of the following embodiment, which will not be repeated here.

[0057] Of course, no matter it is a video in the music video resource library of the application that plays audio or a video generated by the application that plays audio, the first video includes but is not limited to a video displayed in horizontal screen and a video displayed in vertical screen.

[0058] The first control is a user interface (UI) control. Any visual control or element that can be seen on the user interface of the application, such as pictures, input boxes, text boxes, buttons, labels and other controls, some of which respond to user operations. For example, the first control can set the first video as a beautification material in response to the user's operation. The UI controls involved in the embodiments of the present application include but are not limited to: a first control, a confirmation control, a clip selection control, a second control, and the like.

[0059] In some embodiments, the first control is any visual control or element that can be seen on the first interface. The present application does not limit the display position and display type of the first control.

[0060] Beautification material: material that has a beautification effect. The present application embodiment does not limit the effect scene of the beautification material. Optionally, the first video is used as the beautification material to beautify a certain scene.

[0061] Step 220: In response to the operation on the first control, at least two setting options are displayed, and different setting options are used to set the first video as a beautification material for display in different scenes.

[0062] The operation on the first control is an operation for displaying setting options. The embodiment of the present application does not limit the operation type of the operation on the first control. The operation type of the operation on the first control includes but is not limited to single-click, slide, double-click, drag, voice input, face input, etc. Optionally, taking voice input as an example, assuming that the user says "click on the first control" to the terminal device (or client), the terminal device (or client) receives the voice and recognizes the voice content as "click on the first control", then it is considered that the terminal device (or client) has received the operation. In some embodiments, such as Figure 3 As shown, the first control is a control 301 showing "Set as Video Ringtone". Figure 4 As shown, the first control is a control 401 showing "Set as Video Ringtone". Figure 5 As shown, the first control is a control 501 showing "Set as Video Ringtone". Figure 6 As shown, the first control is a control 601 showing "Set as Video Ringtone". Figures 3 to 6 As shown, the first control can be set at different positions of the first interface. At this time, the first interface is an interface where the audio and the video corresponding to the audio are in a playing state.

[0063] In some embodiments, Figure 7As shown, the first control is a control 701 displaying "Video Ringtone". At this time, the first interface is the homepage of the user account, and at this time, the first interface is an interface in which the audio and the video corresponding to the audio are in a paused playback state. In response to the operation of the first control, at least one first video is displayed. Further, after clicking on one of the first videos, an interface in which the audio and the video corresponding to the audio are in a playback state is displayed, and the execution continues from step 210. In some embodiments, as Figure 8 As shown, the first control is a control 801 displaying "Video Ringtone". At this time, the first interface is the main interface of the application. In response to the operation of the first control 801, at least one first video is displayed. Further, after clicking on one of the first videos, an interface showing that the audio and the video corresponding to the audio are in a playing state is displayed, and the execution continues from step 210.

[0064] Setting options: options for setting the first video as the beautification material of the scene. In some embodiments, the setting options are UI controls. The embodiments of the present application do not limit the display position and display type of the setting options. In some embodiments, in response to the operation on the first control, at least two setting options are displayed. At this time, the interface can be kept relatively tidy, which is conducive to improving the user experience of users who do not need to set beautification materials. In other embodiments, in the first interface of audio playback, the first video and at least two setting options are displayed. At this time, the first control is not required, and at least two setting options are directly displayed on the first interface. At this time, the speed of setting the beautification material of the scene can be increased, and there is no need to perform operations on the first control, but directly perform operations on the target setting options. In some embodiments, the number of setting options is related to the number of scenes. Optionally, the number of setting options is positively correlated with the number of scenes. Optionally, the number of setting options is equal to the number of scenes.

[0065] In some embodiments, at least two setting options include at least two of the following: a first setting option, used to set the first video as a beautifying material displayed in an incoming call scenario; a second setting option, used to set the first video as a beautifying material displayed in a charging scenario; a third setting option, used to set the first video as a beautifying material displayed in a prompt sound scenario; a fourth setting option, used to set the first video as a beautifying material displayed in a wallpaper scenario; a fifth setting option, used to set the first video as a beautifying material displayed in a ringback tone scenario.

[0066] In some embodiments, Fig. 9 As shown, there are displayed the setting options 901 of the incoming call scene, the setting options 902 of the ring back tone scene, and the setting options 903 of the charging scene. In other embodiments, such as Fig.10As shown, there are displayed setting options 1001 for incoming call scene, setting options 1002 for charging scene, setting options 1003 for prompt sound scene, and setting options 1004 for wallpaper scene. Different setting options are used to set the first video as a beautification material displayed in different scenes. Setting option 1001 for incoming call scene is used to set the first video as a beautification material displayed in the incoming call scene. Setting option 1002 for charging scene is used to set the first video as a beautification material displayed in the charging scene. Setting option 1003 for prompt sound scene is used to set the first video as a beautification material displayed in the prompt sound scene. Setting option 1004 for wallpaper scene is used to set the first video as a beautification material displayed in the wallpaper scene.

[0067] In other embodiments, when a user wants to use the ring back tone function, it is necessary to establish cooperation with the operator. Fig.11 As shown, sub-figure a and sub-figure b show that when the user sets a video ringback tone, an agreement requiring cooperation with the operator pops up, and on the basis of the established cooperation, the video ringback tone can be successfully set.

[0068] The scene in the embodiment of the present application refers to a place where a terminal device or an application on the terminal device needs and can use beautification materials. Specific descriptions are given below for different scenes.

[0069] In some embodiments, the scene is an incoming call scene. In some embodiments, when other terminal devices call the current terminal device, or the client of application a on other terminal devices calls the client of application a on the current terminal device, there will be an incoming call prompt sound effect and an incoming call prompt interface on the current terminal device. However, conventional incoming call prompt sound effects and incoming call prompt interfaces are relatively simple, so the needs of personalized customization can be met by customizing audio or displaying interfaces. Optionally, at this time, the first video can be applied to the prompt sound effect or prompt interface in the incoming call scene. For how the specific first video is applied to the prompt sound effect or prompt interface in the incoming call scene, refer to the explanation of the following embodiment, which will not be repeated here.

[0070] In some embodiments, the scene is a charging scene. In some embodiments, when the user charges the terminal device, a charging prompt interface or a charging prompt sound effect will be displayed on the terminal device, or a charging special effect will be displayed on the element representing the power on the UI interface. At this time, the charging prompt interface, charging prompt sound effect or charging special effect can also be customized to meet the user's personalized customization needs. For how the specific first video is applied to the prompt sound effect or prompt interface in the incoming call scenario, please refer to the explanation of the following embodiment, which will not be repeated here.

[0071] In some embodiments, the scene is a prompt sound scene. In some embodiments, the terminal device has different prompt sounds for different situations, such as receiving a text message, receiving an application push, and starting up the machine. For these scenes that require prompt sounds, the technical method provided in the embodiment of the present application can customize prompt sounds based on the first video for these scenes, or further, not only have prompt sounds, but also design a prompt interface corresponding to the prompt sounds, which is not limited in the present application.

[0072] In some embodiments, the scene is a wallpaper scene. Optionally, it is considered that the wallpaper can be applied to any place in the terminal device where a picture is needed. For example, the display screen of the main screen of the terminal device needs wallpaper, the background screen of the b application of the terminal device needs wallpaper, the shutdown screen of the terminal device needs wallpaper, the startup screen of the terminal device needs wallpaper, the lock screen interface of the terminal device needs wallpaper, and so on. These scenes that require wallpaper are considered to be wallpaper scenes. Similarly, the technical method provided in the embodiment of the present application can customize wallpapers based on the first video for these scenes that require wallpaper, or further, not only wallpapers, but also designed playback sound effects corresponding to the wallpaper, which is not limited in the present application.

[0073] In some embodiments, the scene is a ringback tone scene. In some embodiments, when other terminal devices call the current terminal device, or when the client of application a on other terminal devices calls the client of application a on the current terminal device, there will be a dial prompt sound effect on the other terminal devices, the most common of which is "beep beep beep", but this method is relatively simple. Therefore, the technical solution provided in the embodiment of the present application designs a video ringback tone and an audio ringback tone, and determines the ringback tone content based on the first video. Optionally, after terminal device a determines the target ringback tone, when terminal device b calls terminal device a, the target ringback tone set by terminal device a is played in terminal device b.

[0074] Of course, the scenarios in the embodiments of the present application include but are not limited to the above five types. Any place in the terminal device that needs to use beautification materials can be considered as a scenario in the embodiments of the present application.

[0075] Step 230, in response to an operation on a target setting option among the at least two setting options, setting the first video as a beautification material displayed in a target scene.

[0076] In some embodiments, the target setting option is any one of the at least two setting options. In some embodiments, the operation on the target setting option in the at least two setting options is an operation for setting the first video as a beautification material displayed in the target scene, and the embodiment of the present application does not limit the operation type of the operation on the target setting option in the at least two setting options. The operation type of the operation on the target setting option in the at least two setting options includes but is not limited to single click, slide, double click, drag, voice input, face input, etc.

[0077] In some embodiments, in response to the operation on the first setting option, the first video is set as the beautification material displayed in the incoming call scene. In some embodiments, in response to the operation on the second setting option, the first video is set as the beautification material displayed in the charging scene. In some embodiments, in response to the operation on the third setting option, the first video is set as the beautification material displayed in the prompt sound scene. In some embodiments, in response to the operation on the fourth setting option, the first video is set as the beautification material displayed in the wallpaper scene. In some embodiments, in response to the operation on the fifth setting option, the first video is set as the beautification material displayed in the ring back tone scene.

[0078] In other embodiments, when the first video is set as a beautification material, the first video is directly used as the beautification material. Alternatively, the first video is processed to obtain the processed material, and the processed material is used as the beautification material applied to different scenes. Optionally, the processing method for the first video includes but is not limited to the intercepted video clips mentioned in the following embodiments, changing the background music corresponding to the first video, etc. In other embodiments, the processing of the first video also includes capturing the position of the target object in the target video frame in the multiple video frames in the first video, taking the position of the target object as the center, and the size of the beautification material in the scene where the beautification material is required or the size after proportional reduction as the basis, intercepting the target video frame to obtain the intercepted video frame, and finally obtaining multiple intercepted video frames according to the multiple video frames, which are combined into the intercepted video as the beautification material. Among them, the target object can be an object appearing in the first video, and the object can be a person, an animal, a plant, etc. In some embodiments, the terminal device determines the target object by itself. Optionally, there are multiple objects appearing in the first video, and the target object is one of the multiple objects. For example, the target object is the object with the highest frequency of appearance among the multiple objects, or the target object is an object at a key position in the picture, then the terminal device determines the object with the highest frequency of appearance among the multiple objects, or the target object is an object at a key position in the picture as the target object. In other embodiments, in response to the user's object selection operation, the target object is determined from multiple objects, and a captured video centered on the target object is further generated based on the target object. Of course, the user's object selection operation can be that the user clicks on the target object in the first video and considers it as the user's object selection operation, or at least one object identifier (such as an avatar or number) can be displayed below the displayed first video. In response to the object selection operation for the identifier of the target object, the terminal device determines the target object, and further generates a captured video centered on the target object based on the target object. Of course, there can also be multiple target objects. The technical solution provided by the embodiment of the present application provides two ways to determine the target object. On the one hand, the terminal device determines the target object by itself. This step can be completed by the background server before the user needs to set the beautification material, without waiting until the user sets the beautification material to process it, which is conducive to improving the speed of determining the target object and the efficiency of generating the video after interception, thereby reducing the processing overhead of the device. On the other hand, the user determines the target object by himself, which can better meet the user's personalized customization needs, making the determination of the beautification material more flexible and interesting.

[0079] The technical solution provided by the embodiment of the present application is that during the audio playback, the video related to the audio displayed during the audio playback is set as the beautification material, that is, when listening to songs or watching music videos, when you encounter a favorite video clip, you can set the currently playing video as the beautification material with one key, without manually importing the audio that needs to be set as the beautification material as in the related art, thereby simplifying the operation complexity of the beautification material setting and improving the setting efficiency of the beautification material. In addition, by selecting a video as the beautification material in the application that plays the audio, the function of the application that plays the audio is expanded, and the characteristics of the rich multimedia resources in the application that plays the audio are fully utilized, so that the user has more options when selecting the video as the beautification material. In addition, when a certain video is set as the beautification material, multiple setting options are provided, so that the user can choose to set the video as the beautification material displayed in different scenes (such as incoming call scene, charging scene, prompt sound scene, wallpaper scene, ringtone scene, etc.), which fully improves the flexibility of the beautification material setting.

[0080] Please refer to Fig.12 , which shows a flow chart of a method for setting beautification materials provided by another embodiment of the present application. The execution subject of each step of the method can be Figure 1 The terminal device 11 in the implementation environment of the scheme shown, for example, the execution subject of each step can be a client. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "client". The method can include at least one of the following steps (310-350):

[0081] Step 310: In a first interface of audio playback, a first video and a first control for setting the first video as a beautification material are displayed, wherein the first video is a video related to the audio.

[0082] Step 320: In response to the operation on the first control, at least two setting options are displayed, and different setting options are used to set the first video as a beautification material for display in different scenes.

[0083] Step 330 , in response to an operation on a target setting option among the at least two setting options, a setting interface corresponding to the target setting option is displayed, wherein selection items of at least two sub-scenes included in the target scene are displayed in the setting interface.

[0084] In some embodiments, the target scene includes at least two subscenes. Optionally, the subscenes include but are not limited to video subscenes, audio subscenes, and picture subscenes. Optionally, the video subscene can be understood as using the video picture in the first video in the target scene. For example, the complete first video is used in the first video subscene. Optionally, the audio subscene can be understood as the audio related to the first video is used in the target scene. For example, the background music of the first video is used in the first audio subscene. Optionally, the audio subscene can be understood as the picture related to the first video is used in the target scene. For example, the picture of the key frame in the first video is used in the first picture subscene.

[0085] In some embodiments, when the target scene is an incoming call scene, such as Fig.13 As shown, in response to an operation on a setting option 1001 in which the target setting option among the at least two setting options is the incoming call scene, a setting interface corresponding to the target setting option is displayed, and the setting interface displays selection items of at least two sub-scenes included in the target scene. The selection item for the video sub-scene may be the incoming call video 1301. The selection item for the audio sub-scene may be the incoming call ringtone 1302. In some embodiments, Fig.14 As shown, when the target scene is a charging scene, in response to the operation of setting option 1002 in which the target setting option among at least two setting options is a charging scene, the setting interface corresponding to the target setting option is displayed, and the setting interface displays selection items of at least two sub-scenes included in the target scene. The selection item for the video sub-scene may be a charging video 1401. The selection item for the audio sub-scene may be a charging ringtone 1402. In some embodiments, when the target scene is a prompt sound scene, the selection item for the video sub-scene may be a prompt video. The selection item for the audio sub-scene may be a prompt ringtone. In some embodiments, when the target scene is a wallpaper scene, the selection item for the video sub-scene may be a dynamic wallpaper video. The selection item for the audio sub-scene may be a wallpaper ringtone. In some embodiments, when the target scene is a color ringtone scene, the selection item for the video sub-scene may be a video color ringtone. The selection item for the audio sub-scene may be a ringtone color ringtone.

[0086] In some embodiments, the setting interface is an interface for setting beautification materials. Optionally, different setting options correspond to different setting interfaces. In some embodiments, the sub-scene selection items are UI controls, and the embodiments of the present application do not limit the display position and display type of the sub-scene selection items on the setting interface.

[0087] Step 340: In response to an operation on a selection item of a first sub-scene among the at least two sub-scenes, a preview effect of setting the first video as a beautification material to be displayed in the first sub-scene is displayed in a setting interface.

[0088] In some embodiments, the operation of the selection item of the first sub-scene in the at least two sub-scenes is an operation for displaying the beautification material displayed in the first sub-scene, and the embodiment of the present application does not limit the operation type of the operation of the selection item of the first sub-scene in the at least two sub-scenes. The operation type of the operation of the selection item of the first sub-scene in the at least two sub-scenes includes but is not limited to single-click, slide, double-click, drag, voice input, face input, etc. In some embodiments, the preview effect is the effect of using the beautification material in the first sub-scene.

[0089] In some embodiments, when the target scene is a charging scene, the selection item for the video sub-scene may be a charging video 1401. The selection item for the audio sub-scene may be a charging ringtone 1402. Fig.14 As shown, in response to the operation of the selection item 1401 for the first sub-scene of the at least two sub-scenes, a preview effect 1403 of setting the first video as the beautification material displayed in the first sub-scene is displayed in the setting interface. Fig.15 As shown, in response to the operation of the selection item 1502 for the first sub-scene among the at least two sub-scenes, a preview effect 1503 of setting the first video as the beautification material displayed in the first sub-scene is displayed in the setting interface.

[0090] In some other embodiments, there are at least two sub-scene selection items for the wallpaper scene. Fig.16 As shown, in response to an operation on a setting option 1004 in which the target setting option among at least two setting options is a wallpaper scene, a setting interface corresponding to the target setting option is displayed, and the setting interface displays selection items of two sub-scenes included in the target scene. The selection items of the sub-scenes include a lock screen sub-scene 1601 and a desktop sub-scene 1602. In other embodiments, there are also at least two sub-scene selection items for the reminder sound scene. Fig.17 As shown, in response to an operation on setting option 1003 in which the target setting option among at least two setting options is a prompt tone scene, a setting interface corresponding to the target setting option is displayed, and the setting interface displays selection items of three sub-scenes included in the target scene. The selection items of the sub-scenes include an alarm sub-scene 1701, a text message sub-scene 1702, and a notification sub-scene 1703.

[0091] Step 350: In response to an operation on a confirmation control in the setting interface, the first video is set as a beautification material displayed in the first sub-scene.

[0092] In some embodiments, the confirmation control is a UI control, and the embodiments of the present application do not limit the display position and display type of the confirmation control on the setting interface. Fig.15 A confirmation control 1501 is shown.

[0093] In some embodiments, the operation of the confirmation control in the setting interface is an operation for determining that the beautification material is used in the first sub-scene, and the embodiment of the present application does not limit the operation type of the operation of the confirmation control in the setting interface. The operation type of the operation of the confirmation control in the setting interface includes but is not limited to single click, slide, double click, drag, voice input, face input, etc.

[0094] In some embodiments, a segment selection control corresponding to the first video is displayed in a setting interface, and the segment selection control is used to customize the selection of video segments in the first video; based on the operation of the segment selection control, the selected video segment in the first video is determined, and the video segment is used to be set as a beautification material to be displayed in the first sub-scene.

[0095] In some embodiments, the segment selection control is a UI control, and the embodiments of the present application do not limit the display position and display type of the segment selection control on the setting interface. In some embodiments, the complete first video is not used in the target scene, but a video segment is intercepted from the first video as a beautification material for use in the target scene. In some embodiments, the operation on the segment selection control is an operation for selecting a video segment from the first video, and the embodiments of the present application do not limit the operation type of the operation on the segment selection control. The operation type of the operation on the segment selection control includes but is not limited to single-click, slide, double-click, drag, voice input, face input, etc.

[0096] In some embodiments, the segment selection control is a draggable progress bar corresponding to the first video. Optionally, different positions in the progress bar correspond to different timestamps in the first video. Optionally, through the segment selection control, the starting position and the ending position are determined from the progress bar, and the video segment between the timestamp corresponding to the starting position and the timestamp corresponding to the ending position is determined as the video segment in the first video that is custom selected. Optionally, the starting position and the ending position can be changed by the user. Optionally, the video segment is determined from the complete progress bar by dragging the progress bar. Optionally, the video segment is determined by directly inputting the time information of the starting timestamp and the ending timestamp. For example, for the operation of the segment selection control, at least two information input boxes are displayed. The time information of the starting timestamp and the ending timestamp is respectively input in the at least two information input boxes, so as to customize the video segment in the first video that is selected.

[0097] The technical solution provided by the embodiment of the present application can meet the different needs of users by setting at least two sub-scenes. When the user only wants a ringtone, the audio related to the video can be set as the ringtone required by the target scene. When the user wants a video, a video clip can be customized to be determined from the video, and the video clip can be set as the video required by the target scene. Therefore, the present application at least meets the beautification needs of different sub-scenes of the target scene. In addition, since the user can customize the video clip, the user's setting experience is improved while improving the flexibility of setting the beautification material.

[0098] Please refer to Fig.18 , which shows a flow chart of a method for setting beautification materials provided by another embodiment of the present application. The execution subject of each step of the method can be Figure 1 The terminal device 11 in the implementation environment of the scheme shown, for example, the execution subject of each step can be a client. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "client". The method can include at least one of the following steps (410-450):

[0099] Step 410: In a first interface of audio playback, a first video and a first control for setting the first video as a beautification material are displayed, wherein the first video is a video related to the audio.

[0100] In some embodiments, at least one of steps S1 to S3 (not shown in the figure) is included before step 410.

[0101] Step S1: In the second interface of the application that plays audio, a second control for generating a video based on an AI model is displayed.

[0102] In some embodiments, the second control is a UI control, and the embodiment of the present application does not limit the display position and display type of the second control on the second interface. The second interface is considered to be an interface on which the second control is displayed. Optionally, the second interface is a main interface of an application that plays audio. The second control is displayed on the main interface.

[0103] Step S2, in response to an operation on the second control, a video production interface is displayed, wherein the video production interface includes at least two production options, and different production options are used to generate videos using different AI models.

[0104] The video production interface is considered to be an interface capable of producing videos. Optionally, the video production interface displays at least two production options. In some embodiments, the production options are UI controls, and the embodiments of the present application do not limit the display position and display type of the production options on the video production interface. In some embodiments, the production options include but are not limited to a first production option, a second production option, and a third production option. Production interface. In some embodiments, such as Fig.19 As shown, the first video can be produced through the production option 1900, for example, the audio can be added to the video picture through the production option 1900. In some embodiments, Fig. 20 As shown, in response to the operation on the second control 2000, a video production interface is displayed.

[0105] Step S3, in response to an operation on a target production option among at least two production options, generating a first video by using an AI model corresponding to the target production option.

[0106] In some embodiments, the first video is a video generated based on an AI model, and the AI ​​model includes a key frame generation model and a video generation model.

[0107] In some embodiments, the AI ​​model is a machine learning model, including but not limited to a neural network model, a deep learning model, etc. The embodiments of the present application do not limit the specific type of the AI ​​model.

[0108] In some embodiments, generating a first video through an AI model includes the following steps.

[0109] The first step is to obtain input information required to generate the first video.

[0110] The embodiments of the present application divide the input information into at least three types, please refer to the introduction of the following embodiments, which will not be repeated here.

[0111] The second step is to generate at least two key frames based on the input information through the key frame generation model.

[0112] In some embodiments, when the input information is text, the key frame generation model is a text-image model, such as DALL-E2. Optionally, the input is text, and the key frame generation model is used to obtain a picture corresponding to the text, that is, a key frame.

[0113] In some embodiments, the key frame generation model includes an encoding network, a mapping network, and a decoding network.

[0114] The second step can be specifically divided into: inputting the input information into the encoding network, and outputting the embedded representation of the input information through the encoding network; inputting the embedded representation of the input information into the mapping network, and outputting the mapped embedded representation through the mapping network; inputting the mapped embedded representation into the decoding network, and outputting at least two key frames through the decoding network.

[0115] In some embodiments, the encoding network is an encoder, the decoding network is a decoder, and the mapping network is a transformer network. This application does not limit the specific network structure.

[0116] In the third step, at least two key frames are interpolated and super-resolution processed through a video generation model to obtain a first video.

[0117] In some embodiments, at least two key frames are interpolated. Exemplarily, 3 frames of images are obtained through the key frame generation network, and after interpolation processing, 30 frames of images are obtained, which can also be understood as smoothing processing of the image, so that the transition between images is smoother and more reasonable. After obtaining the image after interpolation processing, super-resolution processing is performed on the image. Exemplarily, the original image has a resolution of 56*56, and after super-resolution processing, an image with a resolution of 256*256 is obtained. In other embodiments, after at least two super-resolution processes, the image with a resolution of 256*256 is further subjected to super-resolution processing to obtain an image with a resolution of 728*728. Optionally, the video generation model and the key frame generation model are trained separately. Optionally, the key frame generation model is an unsupervised machine learning model. Optionally, the video generation model is a machine learning model trained in advance.

[0118] like Fig.21 As shown, it shows the AI ​​model architecture. Optionally, the AI ​​model includes a key frame generation model 2100 and a video generation model 2110. Among them, the key frame generation model 2100 includes at least an encoding network, a mapping network and a decoding network. Optionally, the video generation model 2110 includes at least interpolation of key frames and at least two super-resolution processes. Optionally, the input of the key frame generation model 2100 is input information, and the output of the key frame generation model 2100 is at least one key frame. Optionally, the input of the video generation model 2110 is at least one key frame, and the output is the generated first video.

[0119] In some embodiments, the operation for the target production option among the at least two production options is an operation for generating the first video through the AI ​​model corresponding to the target production option. The embodiment of the present application does not limit the operation type for the operation for the target production option among the at least two production options. The operation type for the operation for the target production option among the at least two production options includes but is not limited to single click, slide, double click, drag, voice input, face input, etc. In some embodiments, such as Fig. 20 As shown, in the video production interface 2001, in response to an operation on a target production option 2002 among at least two production options, a first video is generated by the AI ​​model corresponding to the target production option 2002.

[0120] The following is a detailed introduction to the video generation process of three different production options and the corresponding AI models.

[0121] In the first type, the input information includes description information, and the first video is a video generated by an AI model based on the description information.

[0122] In some embodiments, the description information is information input by the user or information selected by the user. Optionally, in response to an operation on the first production option, a text input box is displayed. Optionally, the user can enter the description information of the video he wants to generate in the text input box. Of course, the description information here includes but is not limited to at least one or more of words, words, sentences, pictures, and videos. Optionally, the terminal device obtains the description information in response to the user's input operation of the description information, and generates a video through the AI ​​model. Optionally, in response to an operation on the first production option, at least one text tag or video tag is displayed. Optionally, the user can select the tag he wants from at least one text tag or video tag as the description information. Optionally, the terminal device determines the description information in response to the user's tag selection operation, and generates a video through the AI ​​model.

[0123] The second type is that the input information includes the selected style and reference videos and / or reference pictures, and the first video is a video generated by an AI model based on the reference video and / or reference pictures and combined with the selected style.

[0124] In some embodiments, the reference video or reference picture can be uploaded to the client by the user, or can be directly selected by the user from the music video resource library in the client or the video generated by the client. In addition, the user can also edit the video in the client to obtain the reference video or draw the picture to obtain the reference picture. This application does not limit the source of the reference video and reference picture. In other embodiments, in addition to the reference video and reference picture, the user can also select the video style. In other embodiments, after selecting the reference video or reference picture, the style selection interface jumps to the style selection interface, and the user can select or enter his own style in the style selection interface. The terminal device responds to the user's style selection or input operation to determine the style selected by the user. Of course, it is also possible to generate a video through an AI model based only on the reference video or reference picture, and this application does not limit this.

[0125] In some embodiments, the style selected by the user is used as input text, and a reference video and / or a reference picture are used as input conditions to obtain a first video.

[0126] The third type, the input information includes the modification request and the reference video and / or reference picture, and the first video is a video generated by the AI ​​model based on the reference video and / or reference picture and combined with the modification request.

[0127] For the explanation of the reference video and reference picture, please refer to the above embodiment, which will not be repeated here. In some embodiments, the modification request is a modification request for the reference video and / or reference picture proposed by the user. For example, the reference video or reference picture is of a boy, but the character in the video that the user wants to generate is a girl. At this time, the modification request can be "change the boy to a girl". Optionally, the modification request can be input by the user, and the terminal device recognizes the user's input information to obtain the user's modification request. Optionally, the modification request can be selected by the user, and the user interface of the terminal device displays modification labels for different modification directions. In response to the user's selection operation for the label under different modification directions, the user's modification request is determined.

[0128] In some embodiments, the user's modification request is used as input text, and a reference video and / or a reference picture are used as input conditions to obtain a first video.

[0129] In some embodiments, Fig. 22 As shown, it exemplarily shows another AI model model architecture 2200. Optionally, the AI ​​model includes at least a key frame prediction module, a video segment generation module, and a video combination module. In some embodiments, Fig. 22The AI ​​model in is used for automatic generation of videos in the client. Unlike the above, which requires the user to manually input information, the first video is generated by the client itself. Optionally, when you want to make a short video corresponding to a certain audio, determine a song as the audio, and input the lyrics of the song into the AI ​​model. The AI ​​model first divides the lyrics into segments, and extracts elements and labels from the divided audio segments to obtain audio information. For example, when the audio is a song, the lyrics information of the song is extracted as audio information. Optionally, a knowledge base of audio information and image type is constructed, that is, a correspondence between audio information and images is established. Optionally, a picture is determined from the audio information and image type knowledge base according to the audio information as a key frame. Optionally, a video clip is generated based on the selected multiple key frames. Further, a combination method is selected from the constructed video clip combination method knowledge base, and the video clips are combined to obtain a video.

[0130] Of course, when the application playing audio automatically generates the first video, in addition to Fig. 22 In addition to the AI ​​model shown, the first video can be generated by a video generation algorithm instead of an AI model. The following is an exemplary description of the video generation algorithm. Optionally, the complete lyrics of the song selected by the user are displayed on the terminal device, and in response to the user's lyrics selection operation, an audio clip is intercepted, the audio clip corresponds to at least one sentence of lyrics, and the lyrics corresponding to the audio clip are used as audio information. Optionally, the lyrics included in the audio information may belong to multiple songs. Optionally, the lyrics included in the audio information may be discontinuous or continuous. This application does not limit the specific type of audio information. Optionally, an audio information and picture type knowledge base is set in the application that plays audio. Optionally, at least one picture is selected from the audio information and picture type knowledge base according to the audio information selected by the user. Optionally, a video is generated based on the at least one picture. For example, the target picture in the at least one picture is displayed in n time units, and a video with a duration of N is finally obtained, where N is a positive number greater than n, and n is a positive integer. Exemplarily, the audio information and picture type knowledge base is constructed as follows: the picture of the singer of the song to which the lyrics in the audio information belong is used as the picture corresponding to the lyrics. Optionally, one line of lyrics can correspond to at least one picture. Optionally, when the lyrics in the audio information are determined, the picture of the singer corresponding to the line of lyrics is found from the audio information and picture type knowledge base. At this time, the video generation algorithm can be set in the client or in the server, and this application does not limit this. When generating videos based on the video generation algorithm, the use of AI models can be avoided, and the cumbersome model design and training process can be avoided, which reduces the processing overhead of the device to a certain extent.

[0131] Step 420: In response to the operation on the first control, at least two setting options are displayed, and different setting options are used to set the first video as a beautification material for display in different scenes.

[0132] Step 430, in response to an operation on a target setting option among the at least two setting options, setting the first video as a beautification material displayed in a target scene.

[0133] Step 440: Display an audio setting control, where the audio setting control is used to customize the background music of the selected video clip in the first video.

[0134] In some embodiments, the audio setting control is a UI control, and the embodiment of the present application does not limit the display position and display type of the audio setting control. Optionally, the audio setting control is displayed in the setting interface. Of course, the audio setting control can also be displayed in other interfaces.

[0135] In some embodiments, the first video corresponds to audio. Optionally, the video clip has audio corresponding to the video screen of the video clip, and the audio corresponding to the video screen of the video clip is also used as beautification material. In some embodiments, the video clip only provides the video screen but not the audio. Optionally, the audio is determined for the video screen provided by the video clip through the audio setting control.

[0136] Step 450, based on the operation on the audio setting control, determine the background music of the video clip, the background music of the video clip is used to be set together with the video screen of the video clip as a beautification material displayed in the target scene.

[0137] In some embodiments, the operation on the audio setting control is an operation for determining the background music of the video clip, and the embodiment of the present application does not limit the operation type of the operation on the audio setting control. The operation type of the operation on the audio setting control includes but is not limited to single click, slide, double click, drag, voice input, face input, etc.

[0138] In some embodiments, the source of the audio is audio related to the first video. Optionally, the background music of the first video is the first audio. Optionally, based on the operation of the audio setting control, the background music of the video clip is determined from the first audio. In some embodiments, the first video is a 5-minute video, and the background music corresponding to the first video is also 5 minutes. At this time, the video clip intercepted is the video clip from the 1st minute to the 2nd minute. At this time, an audio clip is randomly intercepted from the 5-minute background music as the background music of the video clip. Optionally, the duration of the audio clip is the same as the duration of the video clip. Optionally, the duration of the audio clip can be shorter than the duration of the video clip. Of course, in the part where the duration of the video clip exceeds the duration of the audio clip, the audio clip is repeated or supplemented with blank sound. In some embodiments, a clip with a stronger rhythm is intercepted from the first audio as the background music of the video clip. In some embodiments, a clip of the climax part is intercepted from the first audio as the background music of the video clip. When the climax part of the first audio is from the 3rd to the 4th minute, the audio from the 3rd to the 4th minute is used as the background music of the video clip from the 1st minute to the 2nd minute.

[0139] In other embodiments, the source of the audio is an audio that is unrelated to the first video. Optionally, the background music of the first video is the first audio. Optionally, a second audio is determined from a plurality of audios, and the background music of the video clip is determined based on the operation of the audio setting control. The second audio is another audio different from the first audio, and the background music of the video clip is determined from the second audio. Optionally, the user can enter the name of the audio by himself, so as to obtain the audio setting control corresponding to the second audio. In some embodiments, the first video is a video with a duration of 5 minutes, and the video clip intercepted at this time is a video clip from the 1st minute to the 2nd minute. At this time, an audio clip is randomly intercepted from the second audio as the background music of the video clip. Optionally, the duration of the audio clip is the same as the duration of the video clip. Optionally, the duration of the audio clip can be shorter than the duration of the video clip. Of course, in the part where the duration of the video clip exceeds the duration of the audio clip, the audio clip is repeated or supplemented with a blank sound. In some embodiments, a clip with a stronger rhythm is intercepted from the second audio as the background music of the video clip. In some embodiments, a clip of the climax part is intercepted from the second audio as the background music of the video clip. For example, the audio from the 1st minute to the 2nd minute of the second audio is used as an audio segment and set as the background music of the video segment. Of course, the audio of other timestamps in the second audio can also be intercepted as an audio segment, for example, the audio from the 4th to the 5th minute of the climax of the audio is used as the audio segment corresponding to the video segment from the 1st minute to the 2nd minute.

[0140] In some embodiments, the audio setting control is a draggable progress bar corresponding to the first audio or the second audio. Optionally, different positions in the progress bar correspond to different timestamps in the first audio or the second audio. Optionally, through the audio setting control, the starting position and the ending position are determined from the progress bar, and the audio segment between the timestamp corresponding to the starting position and the timestamp corresponding to the ending position is determined as the music segment selected as the background music in the first video. Optionally, the starting position and the ending position can be changed by the user. Optionally, the audio segment is determined from the complete progress bar by dragging the progress bar. Optionally, the audio segment is determined by directly inputting the time information of the starting timestamp and the ending timestamp. For example, for the operation of the audio setting control, at least two information input boxes are displayed. The time information of the starting timestamp and the ending timestamp are respectively input in the at least two information input boxes, so as to customize the selected audio segment of the first audio or the second audio.

[0141] In some embodiments, the technical solution provided by the embodiments of the present application also includes at least one step among steps S4 to S6 (not shown in the figure).

[0142] Step S4: displaying selection items corresponding to different scenes in the third interface of the audio playing application.

[0143] In some embodiments, the options corresponding to different scenes are UI controls. The embodiments of the present application do not limit the display position and display type of the options corresponding to different scenes. Optionally, the options corresponding to different scenes are displayed on the third interface. Of course, the options corresponding to different scenes can also be displayed on other interfaces. The third interface is considered to be an interface that displays the options corresponding to different scenes. Optionally, the third interface is the main interface of the application that plays audio. The options corresponding to different scenes are displayed on the main interface. In some embodiments, such as Fig.23 As shown, in the third interface 2300 of the application for playing audio, selection items 2301 corresponding to different scenes are displayed.

[0144] Step S5, in response to the operation of the selection item corresponding to the target scene, the material management interface corresponding to the target scene is displayed, and the material management interface corresponding to the target scene displays a third control for turning on or off the target function, and the target function is the function of displaying beautification materials under the target scene.

[0145] In some embodiments, the operation for the selection item corresponding to the target scene is an operation for displaying the material management interface corresponding to the target scene. The embodiment of the present application does not limit the operation type of the operation for the selection item corresponding to the target scene. The operation type for the operation for the selection item corresponding to the target scene includes but is not limited to single click, slide, double click, drag, voice input, face input, etc.

[0146] In some embodiments, the third control is a UI control, and the embodiments of the present application do not limit the display position and display type of the third control. Optionally, the third control is displayed in the material management interface corresponding to the target scene. Of course, the third control can also be displayed in other interfaces. In some embodiments, the material management interface corresponding to the target scene is considered to be an interface showing the third control for turning on or off the target function. In some embodiments, the material management interfaces corresponding to different target scenes are different. In some embodiments, different scenes correspond to different functions. In some embodiments, the material management interface of the target scene also displays the user history settings as the landscaping material of the target scene.

[0147] In some embodiments, Fig.23 As shown, in response to an operation on a selection item corresponding to a target scene, a material management interface 2304 corresponding to the target scene is displayed, and a third control 2302 for turning on or off a target function is displayed in the material management interface corresponding to the target scene.

[0148] Step S6, in response to the operation on the third control, if the target function is in the on state, the target function is turned off, and the beautification material is not displayed in the target scene after the target function is turned off; or, if the target function is in the off state, the target function is turned on, and the beautification material is displayed in the target scene after the target function is turned on.

[0149] In some embodiments, the operation on the third control is an operation for closing or opening the target function, and the embodiment of the present application does not limit the operation type of the operation on the third control. The operation type of the operation on the third control includes but is not limited to single click, slide, double click, drag, voice input, face input, etc.

[0150] It should be noted that in the embodiment of the present application, the execution subject of using the AI ​​model to generate the first video can be a client, of course, it can also be a server. Taking the server as an example, after the client receives the user's input information, the client sends the user's input information to the client's backend server, and the backend server receives the input information and generates the first video based on the AI ​​model. The backend server sends the generated first video to the client, and the client displays it. Generating the first video through the server can reduce the processing pressure of the client to a certain extent and increase the generation speed.

[0151] The technical solution provided in the embodiment of the present application forms assets with beautification materials selected for different first videos, which is easy for users to manage. When the beautification materials are successfully set and merged into the assets, the user can turn off the switch target function with one click, or replace the beautification materials under the target scene from the assets corresponding to the target scene. The efficiency of material management is improved, and at the same time, the user's needs to switch between different beautification materials for the target scene are met. Furthermore, by setting a third control, it is convenient for users to manage the target function, and functions that users do not need to use can be turned off in time, thereby reducing the processing overhead and resource consumption of the device.

[0152] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0153] Please refer to Fig.24 , which shows a block diagram of a device for setting beautification materials provided by an embodiment of the present application. The device has the function of implementing the above-mentioned method example on the client side, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be the terminal device described above, or it can be set in the terminal device. Fig.24 As shown, the device 2400 may include: an interface display module 2410 , an option display module 2420 and a material setting module 2430 .

[0154] The interface display module 2410 is used to display a first video and a first control for setting the first video as a beautification material in a first interface of audio playback, wherein the first video is a video related to the audio.

[0155] The option display module 2420 is used to display at least two setting options in response to an operation on the first control, and different setting options are used to set the first video as a beautification material displayed in different scenes.

[0156] The material setting module 2430 is used to set the first video as a beautification material displayed in a target scene in response to an operation on a target setting option among the at least two setting options.

[0157] In some embodiments, the at least two setting options include at least two of the following: a first setting option, used to set the first video as a beautification material displayed in an incoming call scenario; a second setting option, used to set the first video as a beautification material displayed in a charging scenario; a third setting option, used to set the first video as a beautification material displayed in a prompt sound scenario; a fourth setting option, used to set the first video as a beautification material displayed in a wallpaper scenario; a fifth setting option, used to set the first video as a beautification material displayed in a ringback tone scenario.

[0158] In some embodiments, the material setting module 2430 is used to display a setting interface corresponding to the target setting option in response to an operation on the target setting option among the at least two setting options, wherein the setting interface displays selection items of at least two sub-scenes included in the target scene.

[0159] The material setting module 2430 is further used to display in the setting interface a preview effect of setting the first video as the beautification material displayed in the first sub-scene in response to an operation on a selection item of the first sub-scene among the at least two sub-scenes.

[0160] The material setting module 2430 is further configured to, in response to an operation on a confirmation control in the setting interface, set the first video as a beautification material displayed in the first sub-scene.

[0161] In some embodiments, the material setting module 2430 is further used to display a segment selection control corresponding to the first video in the setting interface, and the segment selection control is used to customize the selection of a video segment in the first video;

[0162] The material setting module 2430 is further used to determine a selected video segment in the first video based on an operation on the segment selection control, and the video segment is used to be set as a beautification material displayed in the first sub-scene.

[0163] In some embodiments, Fig.25 As shown, the module also includes a control display module 2440 and a music determination module 2450.

[0164] The control display module 2440 is used to display an audio setting control, and the audio setting control is used to customize the background music of the video clip selected in the first video.

[0165] The music determination module 2450 is used to determine the background music of the video clip based on the operation of the audio setting control, and the background music of the video clip is used to be set together with the video screen of the video clip as the beautification material displayed in the target scene.

[0166] In some embodiments, the first video is a video in a music video resource library of an application that plays the audio; or, the first video is a video generated by an application that plays the audio.

[0167] In some embodiments, the first video is a video generated based on an AI model, and the AI ​​model includes a key frame generation model and a video generation model.

[0168] In some embodiments, Fig.25 As shown, the module also includes a video generation module 2460.

[0169] The video generation module 2460 is used to obtain the input information required to generate the first video; generate at least two key frames according to the input information through the key frame generation model; and perform interpolation and super-resolution processing on the at least two key frames through the video generation model to obtain the first video.

[0170] In some embodiments, the key frame generation model includes an encoding network, a mapping network, and a decoding network.

[0171] The video generation module 2460 is used to input the input information into the encoding network, and output the embedded representation of the input information through the encoding network; input the embedded representation of the input information into the mapping network, and output the mapped embedded representation through the mapping network; input the mapped embedded representation into the decoding network, and output the at least two key frames through the decoding network.

[0172] In some embodiments, the input information includes description information, and the first video is a video generated by the AI ​​model based on the description information; or, the input information includes a selected style and a reference video and / or a reference picture, and the first video is a video generated by the AI ​​model based on the reference video and / or the reference picture in combination with the selected style; or, the input information includes a modification request and a reference video and / or a reference picture, and the first video is a video generated by the AI ​​model based on the reference video and / or the reference picture in combination with the modification request.

[0173] In some embodiments, the control display module 2440 is also used to display a second control for generating a video based on the AI ​​model in a second interface of the application that plays the audio.

[0174] The interface display module 2410 is also used to display a video production interface in response to an operation on the second control, wherein the video production interface includes at least two production options, and different production options are used to generate videos using different AI models.

[0175] The video generation module 2460 is further configured to generate the first video in response to an operation on a target production option among the at least two production options by using an AI model corresponding to the target production option.

[0176] In some embodiments, Fig.25 As shown, the module also includes a function setting module 2470.

[0177] The control display module 2440 is further used to display the selection items corresponding to the different scenes in the third interface of the application playing the audio.

[0178] The interface display module 2410 is also used to respond to the operation of the selection item corresponding to the target scene to display the material management interface corresponding to the target scene. The material management interface corresponding to the target scene displays a third control for turning on or off the target function, and the target function is the function of displaying beautification materials under the target scene.

[0179] The function setting module 2470 is used to respond to the operation on the third control. If the target function is in the on state, the target function is turned off, and the beautification material is not displayed in the target scene after the target function is turned off; or, if the target function is in the off state, the target function is turned on, and the beautification material is displayed in the target scene after the target function is turned on.

[0180] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0181] Please refer to Fig.26 , which shows a block diagram of a terminal device 2600 provided in one embodiment of the present application. The terminal device 2600 may be Figure 1 The terminal device 11 in the implementation environment shown is used to implement the method for setting the beautification material provided in the above embodiment. Specifically:

[0182] Typically, the terminal device 2600 includes: a processor 2601 and a memory 2602 .

[0183] The processor 2601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 2601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 2601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 2601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2601 may also include an AI processor for processing computing operations related to machine learning.

[0184] The memory 2602 may include one or more computer-readable storage media, which may be non-transitory. The memory 2602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2602 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned method for setting the beautification material.

[0185] In some embodiments, the terminal device 2600 may further optionally include: a peripheral device interface 2603 and at least one peripheral device. The processor 2601, the memory 2602 and the peripheral device interface 2603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 2603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 2604, a display screen 2605, an audio circuit 2607 and a power supply 2608.

[0186] Those skilled in the art will understand that Fig.26 The structure shown in the figure does not constitute a limitation on the terminal device 2600, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0187] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method for setting the beautification material on the terminal device side is implemented.

[0188] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random Access Memory), SSD (Solid State Drives) or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0189] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a terminal device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the terminal device executes the above-mentioned method for setting beautification materials.

[0190] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0191] It should be noted that when the video in the embodiments of the present application is set as a beautification material displayed in different scenes (such as incoming call scenes, charging scenes, prompt sound scenes, wallpaper scenes, ringtone scenes, etc.), it is authorized and allowed by the user and should comply with legal provisions.

[0192] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for setting beautification materials, characterized in that: The method comprises: In a first interface of audio playback, a first video and a first control for setting the first video as a beautification material are displayed, wherein the first video is a video related to the audio; In response to an operation on the first control, at least two setting options are displayed, and different setting options are used to set the first video as a beautification material displayed in different scenes; In response to an operation on a target setting option among the at least two setting options, setting the first video as a beautification material displayed in a target scene; In response to the processing operation on the first video, a processed first video is obtained, wherein the processed first video is used to be set as a beautification material to be displayed in the target scene; The step of obtaining a processed first video in response to the processing operation on the first video includes: In response to a selection operation for at least one target object, at least two target video frames are respectively intercepted with the position of the at least one target object as the center to obtain at least two intercepted video frames; wherein each of the target video frames is any video frame of the first video, the target video frame includes at least one object, and each of the target objects is determined from the at least one object based on at least one of the following: the frequency of occurrence of the target object in the first video, and the position of the target object in the target video frame; The at least two captured video frames are used as the processed first video.

2. The method according to claim 1, characterized in that The at least two setting options include at least two of the following: A first setting option is used to set the first video as a beautification material displayed in an incoming call scenario; A second setting option is used to set the first video as a beautification material displayed in a charging scene; A third setting option is used to set the first video as a beautification material displayed in the prompt sound scene; A fourth setting option is used to set the first video as a beautification material displayed in the wallpaper scene; The fifth setting option is used to set the first video as the beautification material displayed in the ring back tone scenario.

3. The method according to claim 1, characterized in that The step of setting the first video as a beautification material displayed in a target scene in response to an operation on a target setting option among the at least two setting options comprises: In response to an operation on the target setting option among the at least two setting options, displaying a setting interface corresponding to the target setting option, wherein the setting interface displays selection items of at least two sub-scenes included in the target scene; In response to an operation on a selection item of a first sub-scene among the at least two sub-scenes, displaying in the setting interface a preview effect of setting the first video as a beautification material displayed in the first sub-scene; In response to an operation on a confirmation control in the setting interface, the first video is set as a beautification material displayed in the first sub-scene.

4. The method according to claim 3, characterized in that The method further comprises: Displaying a segment selection control corresponding to the first video in the setting interface, the segment selection control being used to customize the selection of a video segment in the first video; Based on the operation on the segment selection control, a selected video segment in the first video is determined, and the video segment is used to be set as a beautification material displayed in the first sub-scene.

5. The method according to claim 1, characterized in that The method further comprises: Displaying an audio setting control, wherein the audio setting control is used to customize the background music of the video clip selected in the first video; Based on the operation on the audio setting control, the background music of the video clip is determined, and the background music of the video clip is used to be set together with the video screen of the video clip as a beautification material displayed in the target scene.

6. The method according to claim 1, characterized in that The first video is a video in a music video resource library of the application that plays the audio; or, The first video is a video generated by an application that plays the audio.

7. The method according to claim 1, characterized in that The first video is a video generated based on an artificial intelligence (AI) model, wherein the AI ​​model includes a key frame generation model and a video generation model; the method further includes: Obtaining input information required to generate the first video; Generate at least two key frames according to the input information by the key frame generation model; The at least two key frames are interpolated and super-resolution processed by the video generation model to obtain the first video.

8. The method according to claim 7, characterized in that The key frame generation model includes an encoding network, a mapping network and a decoding network; The step of generating at least two key frames according to the feature information of the input information by using the key frame generation model includes: Inputting the input information into the encoding network, and outputting an embedded representation of the input information through the encoding network; Inputting the embedded representation of the input information into the mapping network, and outputting the mapped embedded representation through the mapping network; The mapped embedded representation is input into the decoding network, and the at least two key frames are output through the decoding network.

9. The method according to claim 7, characterized in that: The input information includes description information, and the first video is a video generated by the AI ​​model based on the description information; or, The input information includes a selected style and a reference video and / or a reference picture, and the first video is a video generated by the AI ​​model based on the reference video and / or the reference picture and in combination with the selected style; or, The input information includes a modification request and a reference video and / or a reference picture. The first video is a video generated by the AI ​​model based on the reference video and / or the reference picture and combined with the modification request.

10. The method according to claim 7, characterized in that The method further comprises: In a second interface of the application that plays the audio, display a second control for generating a video based on the AI ​​model; In response to an operation on the second control, displaying a video production interface, wherein the video production interface includes at least two production options, and different production options are used to generate videos using AI models with different functions; In response to an operation on a target production option among the at least two production options, the first video is generated by an AI model corresponding to the target production option.

11. The method according to claim 1, characterized in that: The method further comprises: In a third interface of the application playing the audio, selection items corresponding to the different scenarios are displayed; In response to an operation on a selection item corresponding to the target scene, a material management interface corresponding to the target scene is displayed, wherein a third control for turning on or off a target function is displayed in the material management interface corresponding to the target scene, wherein the target function is a function of displaying beautified materials under the target scene; In response to an operation on the third control, if the target function is in an on state, the target function is turned off, and after the target function is turned off, no beautification material is displayed in the target scene; or, if the target function is in an off state, the target function is turned on, and after the target function is turned on, the beautification material is displayed in the target scene.

12. A device for setting beautification materials, characterized in that: The device comprises: An interface display module, used for displaying a first video and a first control for setting the first video as a beautification material in a first interface of audio playback, wherein the first video is a video related to the audio; An option display module, used for displaying at least two setting options in response to an operation on the first control, wherein different setting options are used for setting the first video as a beautification material displayed in different scenes; A material setting module, configured to set the first video as a beautification material displayed in a target scene in response to an operation on a target setting option among the at least two setting options; The material setting module is further used to obtain a processed first video in response to a processing operation on the first video, wherein the processed first video is used to be set as a beautification material displayed in the target scene; The material setting module is also used to respond to a selection operation for at least one target object, and to intercept at least two target video frames respectively with the position of the at least one target object as the center to obtain at least two intercepted video frames; wherein each of the target video frames is any video frame of the first video, and the target video frame includes at least one object, and each of the target objects is determined from the at least one object based on at least one of the following: the frequency of occurrence of the target object in the first video, and the position of the target object in the target video frame; and the at least two intercepted video frames are used as the processed first video.

13. A terminal device, characterized in that: The terminal device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the method according to any one of claims 1 to 11.

Images