Video processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a video processing method and device, a storage medium and a program product. In the method, a service platform can provide a service webpage, and a time axis manager and a video editor are embedded in the service webpage; a user can perform time axis editing on the original material based on the display area corresponding to the time axis manager, so that the original material is synchronously edited in the display area corresponding to the video editor. On one hand, the user can efficiently edit the video through the service webpage provided by the service platform, so that the performance requirement on the terminal equipment is reduced; and on the other hand, through cooperation between the time axis manager and the video editor, the user can perform time axis editing more efficiently based on the time axis manager so as to perform synchronous editing operation in the display area corresponding to the video editor more efficiently and view the editing state and the editing result of the edited original material more intuitively, and the user experience is improved. And the video editing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a video processing method, device, storage medium and program product. Background Art

[0002] With the popularization of mobile terminals and the acceleration of the Internet, short, fast and high-traffic content has gradually gained the favor of major platforms and capital, and people are more willing to use short videos to record and share their lives, or sell goods. Short videos are short videos that integrate topics such as skill sharing, humor, fashion trends, social hot spots, street interviews, public welfare education, advertising creativity, and commercial customization. They are suitable for watching on the move and in a short leisure state. Short videos are widely loved by people for their short, concise and easy-to-spread characteristics.

[0003] In the existing technology, short videos can be operated and edited through powerful video editing software. However, this type of software has a relatively high threshold and is relatively complicated to operate. In addition, high-resolution video editing software also has high requirements for terminal device performance, especially when adding special effects or processing multi-layer videos, which may cause terminal devices to freeze or the editing process to be slow. Summary of the invention

[0004] Multiple aspects of the present application provide a video processing method, device, storage medium and program product for more efficiently editing videos through a service webpage provided by a service platform.

[0005] An embodiment of the present application provides a video processing method, which is applied to a service webpage provided by a service platform, wherein the service webpage is embedded with a timeline manager and a video editor, and the timeline manager and the video editor interact with each other using structured data; wherein the service webpage runs on a terminal device, and the method comprises: displaying the service webpage, wherein the service webpage comprises a first display area corresponding to the video editor, and a second display area corresponding to the timeline manager; the first display area is used to display the original material to be edited, and the second display area is used to display the timeline corresponding to the original material, and different types of original materials correspond to different timelines; in response to a material loading operation, the original material is loaded and displayed in the first display area; and in the second display area, there is no original material corresponding to the timeline. In the case of a corresponding timeline, a timeline is generated according to the type of the original material, and the timeline is displayed in the second display area; the timeline manager is called to respond to the editing operation of the first timeline in the second display area, generate structured editing data, and transmit the structured editing data to the video editor, the structured editing data including identification information of the first timeline, static description information corresponding to the first original material corresponding to the first timeline, and editing actions on the first original material; the video editor is called to edit the first original material according to the static description information and editing actions in the structured editing data, and the editing status and editing results of the first original material are displayed in the first display area, and a target video is generated according to the editing results of multiple original materials.

[0006] Optionally, the service webpage also includes a third display area; the third display area is used for users to load multiple original materials to be edited; in response to a material loading operation, the original material is loaded, including: in response to a material upload operation initiated by the user through an upload entrance in the third display area, the uploaded material is used as the original material; and / or in response to a user selection operation of a candidate material displayed in the third display area, the selected candidate material is used as the original material.

[0007] Optionally, generating a timeline according to the type of the original material includes: generating track lines according to a default timeline style; determining a background color of the track lines according to the type of the original material; adding the background color to the track line axis to obtain a timeline; wherein different types of original material correspond to different background colors.

[0008] Optionally, displaying the timeline in the second display area includes: determining a first position corresponding to the horizontal time baseline of the second display area based on the start time of the original material; determining a second position corresponding to the timeline on the vertical position baseline of the second display area based on the hierarchical relationship or positional relationship between the original material and existing original materials in the target video; taking the intersection of the first position and the second position as the start position of the timeline, and displaying the timeline in the second display area; wherein the positional relationship of each timeline in the second display area on the vertical position baseline represents the hierarchical relationship or positional relationship of the corresponding each original material in the target video.

[0009] Optionally, in response to an editing operation on the first timeline in the second display area, structured editing data is generated, including: in response to the editing operation on the first timeline, highlighting the first timeline; generating an editing action for the first original material based on type information of the editing operation; generating static description information corresponding to the first original material based on editing result information of the first timeline; and encapsulating identification information of the first timeline, the static description information and the editing action according to a set structured data structure to obtain the structured editing data.

[0010] Optionally, based on the type information of the editing operation, an editing action for the first original material is generated, including: in response to the editing operation on the first timeline, determining the type of the editing operation based on the trajectory information of the editing operation or the editing tool used; determining the editing action for the first original material based on the type of the editing operation and in combination with the correspondence between the editing operation type and the editing action.

[0011] Optionally, in response to an editing operation on the first timeline, the first timeline is highlighted, including: highlighting the editing effect on the first timeline according to the type of the editing operation, different editing operations correspond to different editing effects; and transmitting the editing effect to the video editor so that the video editor can synchronously display the editing effect during the editing of the first original material.

[0012] Optionally, the editing operation on the first timeline includes: a drag operation, a delete operation, a crop operation, a selection operation, an object adding operation or a zoom operation; wherein the object adding operation refers to adding a target object at a specified position of the first timeline.

[0013] Optionally, it also includes: when the editing operation is a drag operation, based on the trajectory information of the drag operation, monitoring whether the start time or end time of the first time axis overlaps with the end time or start time of other time axes; when overlap occurs, displaying an adsorption effect in the overlapping position or related area of ​​the two overlapping time axes, and providing prompts through auxiliary lines.

[0014] Optionally, it also includes: displaying a drag control at the starting position of the first timeline, the drag control is linked to the first timeline; when the editing operation is a drag operation, responding to the drag operation of the drag control, initiating a drag operation on the first timeline.

[0015] Optionally, the multiple original materials include a promotional video generated based on a video production service provided by a service platform, and the target video is a result video of editing the promotional video.

[0016] Optionally, it also includes: responding to a video generation instruction for a target object, determining the object description information and video script type of the target object; determining a script structure corresponding to the video script type; the script structure includes multiple storyboards; using an artificial intelligence big model to generate the script content of each storyboard based on the object description information and the script structure; the script content includes spoken text and storyboard description information; the storyboard description information is used to describe the screen content of the storyboard; converting the spoken text corresponding to each storyboard into spoken audio; searching from a material library for a material video that matches the storyboard description information of each storyboard and the target object; synthesizing the spoken audio of each storyboard with the corresponding material video into a storyboard video; and according to the script structure, synthesizing the storyboard videos of the multiple storyboards to obtain a promotional video for the target object.

[0017] Optionally, it also includes: responding to the user's video publishing needs, selecting a target playback platform and a target information retention mini-program template from multiple candidate playback platforms and multiple candidate information retention mini-program templates; different information retention mini-program templates correspond to different business types and mini-program identifiers; sending the identifier of the service platform, the target video and the mini-program identifier corresponding to the target information retention mini-program template to the target playback platform, so that the target playback platform can publish the target video; wherein the published target video is mounted with a target mini-program, and the target mini-program is obtained by the target playback platform integrating the mini-program code corresponding to the mini-program identifier into a preset mini-program shell, and the target mini-program is bound to the identifier of the service platform, and is used to collect the information retention information of the first user who watches the target video, and feed it back to the service platform.

[0018] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; wherein the memory is used to: store one or more computer instructions; and the processor is used to execute the one or more computer instructions to: execute the steps in the video processing method.

[0019] The embodiment of the present application also provides a computer-readable storage medium, which enables the processor to implement the steps in the video processing method when the computer program is executed by the processor.

[0020] An embodiment of the present application further provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the steps in the video processing method are executed.

[0021] In this embodiment, the service platform can provide a service webpage, and the service webpage has a timeline manager and a video editor embedded therein; the user can perform timeline editing on the original material based on the display area corresponding to the timeline manager, thereby synchronously editing the original material in the display area corresponding to the video editor. On the one hand, the user can perform video editing more efficiently through the service webpage provided by the service platform, reducing the performance requirements for the terminal device; on the other hand, through the cooperation between the timeline manager and the video editor, the user can perform timeline editing more efficiently based on the timeline manager, so as to synchronize the editing operation more efficiently in the display area corresponding to the video editor, and more intuitively view the editing status and editing results of the edited original material, thereby improving the efficiency of video editing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 A flowchart of a video processing method provided by an exemplary embodiment of the present application;

[0024] Figure 2 A schematic diagram of the interface of a service webpage provided for an exemplary embodiment of the present application;

[0025] Figure 3 A schematic diagram of a drag effect corresponding to a drag operation provided by an exemplary embodiment of the present application;

[0026] Figure 4a-4b A schematic diagram of a small program template for leaving funds provided for an exemplary embodiment of the present application;

[0027] Figure 5a-Figure 5b A schematic diagram of an information configuration interface provided for an exemplary embodiment of the present application;

[0028] Figure 6 A schematic diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0030] In the video editing scenario, users can usually use powerful video editing software to operate and edit short videos. However, this type of software has a relatively high threshold and is relatively complicated to operate. In addition, high-resolution video editing software also has high requirements for terminal device performance, especially when adding special effects or processing multi-layer videos, which may cause terminal devices to freeze or the editing process to be slow.

[0031] In response to the technical problems existing in the above-mentioned prior art, a solution is provided in some embodiments of the present application. The service platform can provide a service web page, which has a timeline manager and a video editor embedded in it. The user can perform timeline editing on the original material based on the display area corresponding to the timeline manager, and thus synchronously edit the original material in the display area corresponding to the video editor.

[0032] On the one hand, users can edit videos more efficiently through the service web pages provided by the service platform, reducing the performance requirements for terminal devices; on the other hand, through the cooperation between the timeline manager and the video editor, users can edit the timeline more efficiently based on the timeline manager, more efficiently synchronize editing operations in the display area corresponding to the video editor, and more intuitively view the editing status and editing results, thereby improving the efficiency of video editing.

[0033] The technical solutions provided by various embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 The following is a flow chart of a video processing method provided by an exemplary embodiment of the present application. The video processing method can be applied to a service webpage provided by a service platform. The service webpage can be run on a terminal device, which can be any electronic device such as a mobile phone, a tablet computer, and a computer, and is not limited in this embodiment.

[0035] The service webpage has a timeline manager and a video editor embedded in it. The timeline manager is used to manage the timeline corresponding to the material, and the video editor is used to edit the material and generate the corresponding video.

[0036] The timeline manager and the video editor may interact with each other using structured data, where structured data refers to data with a fixed format, which may be written based on any protocol such as JSON (JavaScript Object Notation, a lightweight data exchange format) or XML (eXtensible Markup Language), and this embodiment does not impose any limitation thereto.

[0037] like Figure 1 As shown, the method includes:

[0038] Step 11, display the service webpage, which includes a first display area corresponding to the video editor and a second display area corresponding to the timeline manager; the first display area is used to display the original material to be edited, and the second display area is used to display the timeline corresponding to the original material, and different types of original materials correspond to different timelines.

[0039] Step 12: In response to the material loading operation, the original material is loaded and displayed in the first display area.

[0040] Step 13: If there is no time axis corresponding to the original material in the second display area, generate a time axis according to the type of the original material, and display the time axis in the second display area.

[0041] Step 14, call the timeline manager, respond to the editing operation on the first timeline in the second display area, generate structured editing data, and transmit the structured editing data to the video editor, the structured editing data including identification information of the first timeline, static description information corresponding to the first original material corresponding to the first timeline, and editing actions on the first original material.

[0042] Step 15, calling the video editor to edit the first original material according to the static description information and editing actions in the structured editing data, and display the editing status and editing results of the first original material in the first display area, and generate a target video according to the editing results of multiple original materials.

[0043] In this embodiment, a service web page may be displayed, such as Figure 2As shown, the service webpage may include a first display area corresponding to the video editor and a second display area corresponding to the timeline manager. The first display area is used to display the original material to be edited. The original material can be understood as the video elements required for video generation, such as a certain sticker, a certain text, a certain audio or a certain video, etc. The second display area is used to display the timeline corresponding to the original material, wherein the timeline of the original material is used to organize and display the time sequence and duration of the original material. It should be noted that Figure 2 This is only an exemplary description of the service webpage, and does not limit the relative position relationship between the various display areas of the service webpage and the display size of each display area. The original material may correspond to different types, such as audio type, video type, text type and picture type, etc. Different types of original materials may correspond to different timelines.

[0044] The user can perform a material loading operation according to his / her own needs, and correspondingly, in response to the material loading operation, the original material can be loaded and displayed in the first display area. This embodiment does not limit the specific loading method of the original material, and multiple original materials can be loaded at one time, one original material can be loaded at one time, or one original material can be loaded at one time. This embodiment does not limit this.

[0045] In some cases, the second display area may have a timeline corresponding to the original material. For example, if an original material has been loaded before and corresponding material editing has been performed, after this original material that is not loaded for the first time is displayed in the first display area, the timeline corresponding to the original material can be displayed in the second display area. This timeline can be understood as a timeline that has been drawn previously.

[0046] If there is no timeline corresponding to the original material in the second display area, a timeline can be generated according to the type of the original material and displayed in the second display area. Different types of original materials generate different styles of timelines, so that users can distinguish different types of timelines during video processing and reduce the risk of incorrect editing.

[0047] The user can edit any one of the multiple timelines in the second display area. For the convenience of description below, any one of the multiple timelines is referred to as the first timeline, and the original material corresponding to the first timeline is referred to as the first original material.

[0048] Correspondingly, the timeline manager can be called to respond to the editing operation, generate structured editing data, and transmit the structured editing data to the video editor. The timeline manager can be called by sending a call instruction to the timeline, or by passing parameters, which is not limited in this embodiment. The structured editing data may include identification information of the first timeline, static description information corresponding to the first original material corresponding to the first timeline, and editing actions on the first original material. The identification information of the first timeline is used to identify the identity attributes of the first timeline; the static description information corresponding to the first original material corresponding to the first timeline is used to describe the attributes / features of the first timeline and the first original material, and may include: target state information of the first timeline after editing and identification information and type information of the first original material. The target state information is used to describe the state of the first timeline after editing. The identification information of the first original material is used to identify the identity attributes of the first original material.

[0049] Afterwards, the video editor may be further called. Specifically, the video editor may be called by sending a call instruction to the timeline, or by passing parameters, which is not limited in this embodiment. The video editor may perform an editing operation on the first original material according to the static description information and the editing action in the structured editing data. In other words, the video editor may determine the first original material to be edited according to the structured editing data, and perform a corresponding editing operation on the first original material.

[0050] The video editor can display the editing status and editing results of the first original material in the first display area, and generate a target video according to the editing results of multiple original materials. This embodiment does not limit the specific display method of the editing status of the first original material, which can be any method such as highlight display, flashing display or color change display, and this embodiment does not limit it. After editing multiple original materials, when generating a target video according to the editing results of multiple original materials, the editing results of multiple original materials can be synthesized to obtain the target video.

[0051] By displaying the editing status and editing results of the first original material in the first display area, the user can observe the first original material corresponding to the first timeline in the first display area while editing the first timeline. This can be understood as a collaborative display method of multiple display areas. The user can edit the timeline more efficiently based on the timeline manager, thereby more efficiently synchronizing editing operations in the display area corresponding to the video editor, and more intuitively observe the corresponding editing status and editing effects, thereby improving the efficiency of video editing.

[0052] Based on the above steps 11 to 15, on the one hand, users can perform video editing more efficiently through the service web page provided by the service platform, reducing the performance requirements for terminal devices; on the other hand, through the cooperation between the timeline manager and the video editor, users can perform timeline editing more efficiently based on the timeline manager, so as to more efficiently synchronize editing operations in the display area corresponding to the video editor, and more intuitively view the editing status and editing results of the edited original material, thereby improving the efficiency of video editing.

[0053] In some optional embodiments, such as Figure 2 As shown, the service webpage may also include a third display area, and the third display area may be used for the user to load a variety of original materials to be edited. Based on this, the original materials may be loaded in response to the material loading operation through the following implementations:

[0054] Embodiment 1: In response to a material upload operation initiated by a user through an upload entry in the third display area, the uploaded material is used as the original material. The upload entry can be implemented as an upload control or an upload window, etc., which is not limited in this embodiment. The user can select materials such as pictures, texts or videos from a local or remote disk, a picture library, a video library or a text library through the upload entry for uploading.

[0055] Embodiment 2: In response to a user's selection operation on a candidate material displayed in the third display area, the selected candidate material is used as the original material.

[0056] That is to say, the first embodiment can support users to upload materials as original materials, and the second embodiment can support users to select original materials from existing candidate materials. The first and second embodiments can be executed separately or in combination, and this embodiment does not limit them. In this way, users can upload rich original materials more efficiently through the third display area.

[0057] After the original material is loaded in the above manner, if there is no timeline corresponding to the original material in the second display area, the timeline can be generated according to the type of the original material in the following manner:

[0058] First, track lines are generated according to the default timeline style, wherein Canvas can be called to draw track lines according to the default timeline style. Among them, Canvas is a drawing tool that can execute drawing commands based on JavaScript (a programming language). After that, the background color of the track lines can be determined according to the type of original material. Among them, the background color of the track lines can be determined based on the correspondence between the preset type and the background color of the track lines and the type of original material, and different types of original materials correspond to different background colors. Then, a background color can be added to the track line axis to obtain a timeline. In this way, the type of original material is different, and the background color of the generated timeline is also different, so that users can distinguish different types of timelines during video processing and reduce the risk of incorrect editing.

[0059] In some optional embodiments, after the timeline is generated according to the type of the original material, the timeline may be displayed in the second display area based on the following steps:

[0060] Step R1: Determine the first position corresponding to the horizontal time base of the second display area according to the start time of the original material. The horizontal time base can be used as the time base of each time axis, such as Figure 2 As shown, the line between point A and point B is the horizontal time baseline. Since the start time of the horizontal time baseline is 00:00, the corresponding first position on the horizontal time baseline of the second display area can be determined according to the start time of the original material and the start time of the horizontal time baseline.

[0061] Step R2: Determine the second position corresponding to the time axis on the vertical position baseline of the second display area according to the hierarchical relationship between the original material and the existing original material in the target video. For example, if the hierarchical relationship between the original material and the existing original material in the target video is: the level of the original material is above the level of the existing original material, then the second position corresponding to the time axis of the original material on the vertical position baseline of the second display area is also above the time axis corresponding to the existing original material.

[0062] In addition, the second position corresponding to the time axis on the vertical position baseline of the second display area can also be determined according to the positional relationship between the original material and the existing original material in the target video. For example, if the positional relationship between the original material and the existing original material in the target video is that the original material is displayed below the existing original material, then the second position corresponding to the time axis on the vertical position baseline of the second display area is also below the time axis corresponding to the existing original material.

[0063] Step R3, taking the intersection of the first position and the second position as the starting position of the time axis, and displaying the time axis in the second display area; wherein the positional relationship of each time axis on the vertical position baseline in the second display area represents the hierarchical relationship or positional relationship of the corresponding original materials in the target video.

[0064] Based on the above steps R1 to R3, the starting position of the timeline can be determined more accurately according to the starting time of the original material and the hierarchical relationship or positional relationship between the original material and the existing original material in the target video. Based on this method, the arrangement logic between the various timelines presented in the second display area can correspond to the arrangement logic of the various original materials in the first display area in the target video, so that the user can more intuitively perceive the relationship between the various original materials in the first display area and the various timelines in the first display area.

[0065] On the basis of the above-mentioned embodiments, when generating structured editing data in response to the editing operation on the first time axis in the second display area, the following can be achieved based on step K1 and step K2:

[0066] Step K1: In response to an editing operation on the first timeline, the first timeline is highlighted.

[0067] The embodiments of the present application do not limit the editing operations on the first timeline. In some exemplary embodiments, the editing operations on the first timeline may include: dragging operations, selecting operations, deleting operations, cropping operations, object adding operations, or zooming operations. Among them, the object adding operation refers to adding a target object to a specified position on the first timeline. Among them, the target object may be text, animation, motion effect, picture or control, etc., which is not limited by this embodiment. The embodiments of the present application do not limit the specific implementation method of highlighting, which may be a color change display, a flashing display, a bold border display, or adding a highlight mark on the first timeline, etc., which is not limited by this embodiment.

[0068] Specifically, the editing effect on the first timeline can be highlighted according to the type of editing operation, and different editing operations correspond to different editing effects. For example, if the type of editing operation is a selected type, the corresponding editing effect can be a bold border display; for another example, if the type of editing operation is a zoom type, the corresponding editing effect can be a flashing display.

[0069] Afterwards, the editing effect can be transmitted to the video editor so that the video editor can synchronously display the editing effect during the editing process of the first original material. Specifically, the structured effect data can be determined according to the set structured data structure, the identification information of the first time axis and the editing effect information of the first time axis can be packaged to obtain the effect data, and the effect data can be sent to the video editor. The video editor can synchronously display the editing effect during the editing process of the first original material according to the effect data.

[0070] In this way, the editing effects can be synchronized in the first display area and the second display area for the same edited original material, so that the user can intuitively perceive the original material corresponding to the timeline in the first display area when performing timeline editing in the second display area.

[0071] Step K2, based on the type information of the editing operation, generate an editing action for the first original material. Specifically, in response to the editing operation on the first timeline, the type of the editing operation is determined based on the trajectory information of the editing operation or the editing tool used. Wherein, when determining the type of the editing operation based on the trajectory information of the editing operation, the real-time position corresponding to the editing operation can be monitored to determine the trajectory information of the editing operation based on the monitored real-time position. For example, the trajectory information of the editing operation shows that the editing operation has moved on the horizontal time baseline, that is, the editing operation involves the adjustment of the time point, and the type of the editing operation can be determined as the dragging type. In other words, the user adjusts the time point of the first timeline through the horizontal dragging operation; for another example, the trajectory information of the editing operation shows that the editing operation has moved on the vertical position baseline, that is, the editing operation involves the adjustment of the hierarchical relationship or the positional relationship, and the type of the editing operation can be determined as the dragging type. In other words, the user adjusts the hierarchical relationship or the positional relationship of the first timeline in multiple timelines through the vertical dragging operation.

[0072] The type of the editing operation is determined according to the editing tool used in the editing operation. For example, if the editing operation uses a zoom tool, then the type of the editing operation is a zoom type; for another example, if the editing operation uses a crop tool, then the type of the editing operation is a crop type.

[0073] After determining the type of editing operation, the editing action on the first original material can be determined according to the type of editing operation and the correspondence between the editing operation type and the editing action. Among them, the dragging operation on the horizontal time baseline can correspond to the editing action of adjusting the time point; the dragging on the vertical position baseline can correspond to the editing action of adjusting the hierarchical relationship or the editing action of adjusting the position relationship. The cropping operation can correspond to the cropping action on the first original material. The deleting operation can correspond to the deleting action on the first original material. The scaling operation can correspond to the scaling action on the first original material. The operation of adding a target object can correspond to the action of adding a motion effect or adding a text to the first original material, etc.

[0074] In this way, the editing action on the original material corresponding to the timeline can be determined more accurately based on the editing operation on the timeline.

[0075] Step K3: Generate static description information corresponding to the first original material according to the editing result information of the first time axis. Specifically, the target state information of the first time axis after editing, as well as the identification information and type information of the first original material can be obtained as the static description information.

[0076] Step K4: encapsulate the identification information, static description information and editing actions of the first time axis according to the set structured data structure to obtain structured editing data. The specific implementation of the structured data structure is not limited in this embodiment. In some exemplary embodiments, it can be a JSON data structure or an XML data structure.

[0077] Based on the above steps K1 to K4, structured editing data can be generated more accurately according to the editing operation on the first timeline, so as to achieve synchronous editing of the first original material in the first display area and synchronization of editing effects based on the structured editing data.

[0078] In some optional embodiments, when the editing operation is a drag operation, it is possible to monitor whether the start time or end time of the first time axis coincides with the end time or start time of other time axes according to the track information of the drag operation. Figure 3As shown, when overlap occurs, an adsorption effect can be displayed in the overlap position or related area of ​​the two overlapping time axes, and a prompt auxiliary line can be used for prompting. Among them, the prompt auxiliary line is used to show the overlap effect of the two time axes where the end time overlaps or the start time overlaps. Optionally, a drag control is also displayed at the starting position of the first time axis, wherein the drag control can be linked with the first time axis. The drag control can be implemented as a control to assist the dragger, or as other types of controls, which is not limited in this embodiment. When the editing operation is a drag operation, in response to the drag operation of the drag control, a drag operation is initiated on the first time axis. In this way, the first time axis can be dragged more efficiently by dragging the drag control.

[0079] In some optional embodiments, the various raw materials in the aforementioned embodiments may include a promotional video generated based on a video production service provided by the service platform, and the target video is a result video of editing the promotional video. Specifically, the promotional video may be generated based on the following methods:

[0080] Step F1, responding to a video generation instruction for a target object, determining object description information and a video script type of the target object.

[0081] Among them, the object description information may include the name, characteristics, uses, advantages, etc. of the target object, which can be used as the basis of the video content. For example, the following is the object description information of the cleaning service: "Professional home cleaning service, covering whole house cleaning, including kitchen, bathroom, bedroom and other areas, meticulous service, and use of environmentally friendly detergents". Through the object description information, the name of the target object can be introduced, such as "home cleaning service", where the service content of "home cleaning service" includes "whole house cleaning, including kitchen, bathroom, bedroom and other areas", and the service advantages and characteristics are "meticulous service, use of environmentally friendly detergents". Among them, the video script can be used to guide the elements and links in the video generation process, and different types of video scripts can be used to generate different types of promotional videos. The video script may include multiple storyboards, each of which can be used to guide the generation of one of the video segments of the promotional video, and the video segment can be a link in the process of generating the promotional video. After receiving the video generation instruction, the video script type can be determined according to the target object, the purpose of promotion, etc.; or the user can specify the desired video script type in the video generation instruction. The video script type may include, for example, narrative type, explanatory type, etc., where each video script type corresponds to a different script structure.

[0082] After determining the type of video script, you can determine the script structure corresponding to the type of video script. The script structure can refer to the number of storyboards, and each storyboard can be used to display specific content. Take the three-part script structure as an example. This structure divides the video content into three main parts or storyboards. First, storyboard 1 can focus on emotional marketing, with the goal of establishing a connection with the audience through emotional resonance, stimulating the audience's interest and emotional response; storyboard 2 is dedicated to service introduction, where the characteristics and advantages of the service provided can be introduced in detail to ensure that the audience has a clear understanding of the service itself. Finally, storyboard 3 shows the applicable population of the service and shares the feedback and evaluation of the experiencers, which helps the audience understand the actual application effect of the service and enhances the sense of trust.

[0083] Through this structured approach, the video script can be developed in an orderly manner, with each storyboard designed for specific goals and content, thus forming a coherent and persuasive video narrative. This clear script structure not only helps guide video shooting and production, but also helps ensure that the final video content can effectively convey information, attract target audiences, and achieve established marketing goals.

[0084] Step F2: determine the script structure corresponding to the video script type; the script structure includes multiple storyboards.

[0085] Among them, the oral copy can include the narration or dialogue text in the current storyboard, and the storyboard description information can be used to illustrate the visual content of each storyboard, for example, it can include scene layout, character actions, camera angles, lens movement, etc. The video generation module of artificial intelligence can understand the overall framework of the video and the order of each storyboard based on the script structure, and understand the basis of the script content based on the object description information of the target object, so as to intelligently generate the script content of each storyboard. The storyboard description information can make the generated promotional video picture accurately convey the expected information and emotions, and can also maintain coordination with the oral copy.

[0086] Step F3: Generate the script content of each storyboard based on the object description information and the script structure using the artificial intelligence big model; the script content includes the spoken copy and the storyboard description information; the storyboard description information is used to describe the picture content of the storyboard.

[0087] Step F4: Convert the spoken text corresponding to each storyboard into spoken audio.

[0088] Step F5: Search the material library for material videos that match the storyboard description information of each storyboard and the target object.

[0089] Step F6: Combine the spoken audio of each storyboard with the corresponding source video to form a storyboard video.

[0090] Step F7: According to the script structure, synthesize the storyboard videos of multiple storyboards to obtain a promotional video for the target object.

[0091] After the spoken copy is generated, the spoken copy of each storyboard can be converted into spoken audio using text-to-speech conversion technology. Next, based on the picture content described by the storyboard description information of the storyboard, the material content that matches the picture content described by the storyboard description information and matches the target object can be searched from the material library. Among them, matching the target object can refer to the material content that matches the product corresponding to the target object. By searching for material videos based on the storyboard description information and the target object, it can be ensured that the visual content of the generated promotional video matches the target object and the usage scenario of the target object.

[0092] For the material video and spoken audio corresponding to each storyboard, the corresponding material video and spoken audio can be synthesized and synchronized to generate a target storyboard video. Then, according to the script structure, multiple target storyboard videos can be synthesized in order to generate a promotional video for the target object.

[0093] In some embodiments, the use of an artificial intelligence video generation module to generate the script content for each storyboard based on object description information and a target script structure can be specifically implemented as follows: the use of an artificial intelligence video generation module to generate a spoken text for each storyboard based on object description information and a target script structure; the use of an artificial intelligence video generation module to generate storyboard description information for each storyboard based on the spoken text for each storyboard.

[0094] Among them, the video generation module of artificial intelligence can integrate the object description information into the script according to the provided object description information and the target script structure through understanding the object description information, and generate a spoken copy for each storyboard that conforms to the script structure and the target object. The video generation module of artificial intelligence can generate storyboard description information for describing the visual content corresponding to the spoken copy through understanding the spoken copy, so that the visual effects presented in the generated promotional video match the spoken content. In this way, high-quality script content can be efficiently generated, and the copy in the script content is closely integrated with the visual effects.

[0095] In some embodiments, the video generation module of artificial intelligence is used to generate the oral copy of each storyboard based on the object description information and the target script structure, which can be specifically implemented as follows: obtaining the target script template and the target script requirements corresponding to the target script structure; generating the first prompt word according to the first prompt template according to the object description information, the target script template and the target script requirements; inputting the first prompt word into the video generation module of artificial intelligence, so as to use the video generation module of artificial intelligence to generate the oral copy of each storyboard based on the object description information, the target script template and the script requirements. Among them, the target script template can refer to a script pre-generated under the target script structure, which includes the same number of storyboards as the target script structure, and each storyboard includes the corresponding script content. For example, the target script template can be generated by reverse parsing other popular promotional videos that have been generated. Thus, by generating the script content according to the target script template generated by reverse parsing other popular promotional videos, the script structure, script style, etc. can be imitated to improve the maturity and popularity of the generated script content.

[0096] The target script requirements can be used to indicate the length, style, target audience, content requirements, format requirements, etc. of the script expected to be generated based on the target script structure. The first prompt template can be a predefined framework or structure for guiding and standardizing the content input to the video generation module of the artificial intelligence. In an embodiment of the present application, the first prompt template may include elements such as script structure, target audience, copy length, format requirements, language and tone, specific requirements, etc. The first prompt word can be generated by filling the object description information, the target script template, and the target script requirements into the corresponding elements of the first prompt template. By inputting the first prompt word into the video generation module of the artificial intelligence, the video generation module of the artificial intelligence can generate the oral copy of each storyboard based on the object description information, with reference to the target script template and according to the script requirements.

[0097] In some embodiments, based on the spoken audio of each storyboard, a digital human image of the digital human broadcasting the spoken audio can also be generated.

[0098] Among them, Digital Human or Virtual Human can be a computer-generated character with human-like appearance, behavior and communication ability. Digital Human can be two-dimensional or three-dimensional and can be created through computer graphics (CGI) and artificial intelligence technology. Digital Human screen can be used to show the facial movements of Digital Human that match the spoken audio, such as facial expressions and lip shape.

[0099] In some embodiments, synthesizing the spoken audio of each storyboard with the corresponding target material video into a target storyboard video can be specifically implemented as follows: synthesizing the spoken audio of each storyboard with the digital human screen into a digital human video; in the case where a target material video exists for any storyboard, synthesizing the digital human video and the target material video into a target storyboard video; in the case where a target material video does not exist for any storyboard, using the digital human video as the target storyboard video.

[0100] After synthesizing the spoken audio and the digital human image into a digital human video, the digital human can broadcast the spoken audio according to facial movements that match the spoken audio and use corresponding timbre, intonation and emotional expression, so that the visual and auditory elements of the digital human video are coordinated.

[0101] If the target material video exists in any storyboard, the digital human video can be added to the upper layer of the target material video layer, so that the digital human image is integrated into the target material video, and the target material video is used as the background for the digital human image to perform voice broadcasting, thereby improving the display effect. If the target material video does not exist in any storyboard, the digital human video can be used independently as the video content of the storyboard. In this case, appropriate background elements can be added to the digital human video to ensure the richness and visual appeal of the picture.

[0102] In some embodiments, it is also possible to obtain digital person selection prompt information of multiple digital persons; and determine the target digital person selected by the user according to the digital person selection request sent by the user. The digital person selection prompt information may include a brief introduction, characteristics, appearance and other information of each digital person among the multiple digital persons. Thus, after receiving the digital person selection prompt information, the user may send a digital person selection request according to preferences or specific purposes. The digital person selection request may include identification information of the digital person selected by the user. The service platform may respond to the digital person selection request and may determine the target digital person selected by the user according to the information in the digital person selection request.

[0103] In some embodiments, based on the spoken audio of each storyboard, generating a digital human screen of a digital human broadcasting the spoken audio can be specifically implemented as follows: based on the spoken audio, generating the lip shape features of the target digital human; synthesizing the lip shape features into the digital human model corresponding to the target digital human, and generating the digital human screen. The service platform can analyze the spoken audio to determine the rhythm, intonation and pronunciation of the speech, thereby generating lip shape features synchronized with the audio. Lip shape features can include, for example, the opening and closing of the lips, the position of the tongue, and possible accompanying facial expressions.

[0104] In a possible implementation of the present application, speech recognition and analysis technology can be used to decompose the audio signal of the oral audio into individual factors and syllables. Then, an artificial intelligence model (i.e., the artificial intelligence-based video generation module mentioned in this embodiment) can be used to map the decomposed factors and syllables to specific lip movements. Among them, the artificial intelligence model can, for example, be based on a machine learning algorithm and have the ability to identify the lip shape, tongue position, and facial expression corresponding to different factors. After the artificial intelligence model identifies the lip shape corresponding to each factor, these lip movements can be synthesized using technologies such as facial animation to create a coherent lip animation sequence. In this process, it is necessary to accurately synchronize the lip shape of the digital person with the pronunciation in the oral audio to achieve a natural and smooth visual effect. In addition, in order to enhance the expressive ability of the digital person, the service platform can also analyze the emotions and intonation in the audio, and integrate these emotional factors into the lip shape and facial expressions, so that the digital person can convey information more vividly. Finally, the generated lip shape features and facial animations can be integrated into the target digital human model, and the digital human screen can be generated by real-time rendering technology to ensure that the lip shape, facial expression, and overall performance of the digital person match the audio content when the voice is broadcast.

[0105] In some embodiments, synthesizing the digital human video and the target material video into the target storyboard video may be specifically implemented by replacing the background image in the digital human video with the material image of the target material video to generate the target storyboard video.

[0106] In some embodiments, at least one object video input by the user can also be obtained; each object video is segmented according to the content similarity to obtain multiple segmented videos; the multiple segmented videos are converted into video features respectively using the video generation module of artificial intelligence; the multiple segmented videos are respectively used as material videos, and are saved in the material library in correspondence with their respective video features and the object type corresponding to the object video. Among them, the object video can be a video associated with the target object shot by the user, generated by specific software, or downloaded from the Internet. The object video may be long in length, and the object video may display multiple different themes. Therefore, in order to increase the number and richness of the material videos when generating promotional videos, the object video provided by the user can be split. By splitting the object video, the longer object video can be divided into multiple shorter material videos according to different themes or contents. The service platform can determine the split position of the object video by identifying the key frames, scene changes or other important content points in the object video, and split the object video at each split position to obtain multiple segmented videos.

[0107] In a possible implementation, the service platform can perform frame extraction processing on the object video to extract a series of separate video frames from the object video. The service platform can perform similarity calculation between consecutive video frames, and calculate the similarity between consecutive video frames by comparing the visual content between consecutive frames, such as color, texture, shape, and motion features. Among them, the similarity calculation can be calculated by, for example, pixel comparison, feature vector matching, or the output of a deep learning model. According to the calculated similarity results, the service platform can determine the splitting point of the object video, that is, the key frame. The splitting point can be located at a position where the similarity drops significantly, which can indicate that the video content is changing or switching to a new scene. Thus, the service platform can split the object video into multiple segmented videos on these key frames, and each segmented video can contain a series of visually similar continuous frames, so that the consistency of the content of each segmented video can be guaranteed. In addition, when performing segmentation processing, the service platform can also consider the contextual information and semantic information of the video to improve the accuracy and relevance of the segmentation; for example, if the dialogue or narrative in the video logically belongs to the same subject, it may not be split into different segments even if there is a slight visual change.

[0108] In some embodiments, searching for a material video that matches the storyboard description information of each storyboard and the target object from the material library can be specifically implemented as follows: performing feature extraction processing on the storyboard description information to generate storyboard description features; based on the degree of matching between the storyboard description information of each storyboard and the video features, searching for a target material video that matches each storyboard from a plurality of material videos in the material library that match the object type of the target object. The service platform can analyze the storyboard description information of each storyboard, extract the visual and content features of the storyboard description information, and convert these features into a storyboard description feature vector. The storyboard description feature vector can contain the core elements of the storyboard, such as scenes, characters, actions, colors, and emotions, etc., to provide a basis for searching and matching video materials.

[0109] After generating the storyboard description feature vector, the service platform can match the storyboard description feature vector with the video feature vector in the material library. The video materials in the material library can be pre-analyzed and converted into their own video feature vectors, which can be used to describe the visual content of the video. By calculating the similarity between the storyboard description feature vector and the video feature vector, the service platform can quantify the degree of match between each video material and a specific storyboard.

[0110] When matching video materials, the service platform not only considers the matching degree of feature vectors, but also searches for video materials that are consistent with the type of the target object in the classified material videos, and selects matching material videos from the material library in combination with the object type of the target object.

[0111] In this way, after a promotional video of the target object is generated for the user more efficiently according to the user's video production needs, the promotional video can be edited accordingly to obtain a result video, which is the target video in the aforementioned embodiment. Afterwards, the target video can be published to the corresponding video playback platform to promote the target object to the user group in the video playback platform. The specific publishing process can be implemented based on the following steps:

[0112] Step H1, in response to the user's publishing needs, select a video playback platform and a retention mini program template corresponding to the target video from multiple candidate playback platforms and multiple candidate retention mini program templates; different retention mini program templates correspond to different business types and mini program identifiers.

[0113] Step H2: Send the service platform's identifier, the target video, and the mini-program identifier corresponding to the information-retention mini-program template to the video playback platform so that the video playback platform can publish the target video; wherein, the published target video is mounted with a target mini-program, and the target mini-program is obtained by the video playback platform integrating the mini-program code corresponding to the mini-program identifier into a preset mini-program shell. The target mini-program is bound to the service platform's identifier, and is used to collect the information retention information of the first user who watches the target video, and feed it back to the service platform.

[0114] The video playback platform can be used to play the target video to promote the target object to the first user, and can be a short video platform, a long video platform, a live broadcast platform, etc., wherein the first user refers to a user who uses the video playback platform to watch the video. The information retention applet template refers to a template of the information retention applet that can be mounted on the playback platform, and can be used to display the interface style of the information retention applet.

[0115] Different information retention mini program templates may correspond to different mini program identifiers and different business types. Among them, the mini program identifier corresponding to the information retention mini program template is used to identify the identity of the information retention mini program template. Different information retention mini program templates may correspond to different business types. Among them, different target objects may belong to different or the same business types respectively. For example, down jacket cleaning services and snow suit dry cleaning services both belong to the business type of laundry services; for another example, designated driver services and car rental services may respectively belong to the business type of transportation services; the above is only an exemplary description, and the specific type classification rules can be customized according to actual needs, and this embodiment does not limit it.

[0116] In this embodiment, different information-reservation mini-program templates may be set for different business types, so that the information-reservation mini-program is more adaptable to the business type of the target object. Figure 4a-4b The following are three examples of the template for the leave a capital app: Figure 4aAs shown, the information leaving applet template L1 corresponding to the laundry service business type may include a submit control of "Submit information and wait for contact" and the following configuration items: relevant pictures of the target object, contact information, required services, clothing types, laundry styles, and door-to-door service addresses. Figure 4b As shown, the information retention applet template L2 corresponding to the business type of transportation services may include a submission control of "Submit information and wait for contact" and the following configuration items: relevant pictures of the target object, contact information, required services, vehicle type, ideal price range, and door-to-door service address. It should be noted that Figure 4a-4b It is only used to provide an illustrative description of the template for the Mini Program for Retaining Information, and does not limit the specific style of the template for the Mini Program for Retaining Information.

[0117] The specific selection method of the video playback platform and the resource-reservation mini-program template is not limited in the embodiments of the present application. In some exemplary embodiments, the video playback platform and the resource-reservation mini-program template corresponding to the target video can be selected from multiple candidate playback platforms and multiple candidate resource-reservation mini-program templates based on the following two implementation methods:

[0118] Implementation method 1: Display an information configuration interface to the user, and display multiple candidate playback platforms and multiple candidate retention app templates on the information configuration interface, such as Figure 5a Among them, displaying each candidate retention mini program template can be understood as displaying the interface thumbnail of the retention mini program corresponding to each candidate retention mini program template.

[0119] In response to the user's first selection operation, a candidate playback platform selected from multiple candidate playback platforms is determined as the target playback platform; in response to the user's second selection operation, a candidate retention mini-program template selected from multiple candidate retention mini-program templates is determined as the retention mini-program template corresponding to the target video.

[0120] Implementation method 2: Figure 5b As shown, a first information configuration interface is displayed to the user, and multiple candidate playback platforms are displayed on the first information configuration interface; in response to a first selection operation, a candidate playback platform selected from the multiple candidate playback platforms is determined as the video playback platform corresponding to the target video. A second information configuration interface is displayed to the user; multiple candidate retention mini-program templates are displayed on the second information configuration interface; in response to a second selection operation, a candidate retention mini-program template selected from the multiple candidate retention mini-program templates is determined as the retention mini-program template corresponding to the target video.

[0121] Implementation method three: determine the target playback platform from multiple candidate playback platforms based on the performance indicator information of multiple candidate playback platforms. Among them, the performance indicator information may include one of the following: network delay information, server load information, traffic information used to promote the target object in the historical period, and historical popularity information of the target object in the historical period. Among them, the traffic information used to promote the target object in the historical period refers to: the traffic indicator data related to the target object counted by the candidate playback platform in the historical period, which may include one of the following: number of views, number of interactions or user engagement, etc. Among them, the historical popularity information of the target object in the historical period refers to the popularity of the target object in the historical period, which can be obtained by comprehensive calculation based on the likes data, comment data, and forwarding data of the target object on the candidate playback platform.

[0122] Among them, based on the network delay information of multiple candidate playback platforms, the candidate playback platform whose network delay meets the preset delay condition can be selected as the video playback platform corresponding to the target video; based on the server load information of multiple candidate playback platforms, the candidate playback platform whose server load meets the preset load condition can be selected as the video playback platform corresponding to the target video; based on the traffic information of multiple candidate playback platforms used to promote target objects in a historical period, the candidate playback platform whose traffic used to promote the target objects in a historical period meets the preset traffic condition can be selected as the video playback platform corresponding to the target video; based on the historical heat information of the target objects of multiple candidate playback platforms in a historical period, the candidate playback platform whose historical heat meets the preset heat condition can be selected as the video playback platform corresponding to the target video.

[0123] In this implementation, the target business type corresponding to the target object can be determined based on the object description information. Specifically, the object description information can be semantically understood based on the semantic understanding module of artificial intelligence to obtain the target business type corresponding to the target object. Furthermore, a candidate retention mini program template corresponding to the target business type can be selected from multiple candidate retention mini program templates as the video playback platform corresponding to the target video.

[0124] Through the above-mentioned implementation modes one to three, a method for manual selection by users and a method for automatic selection by video processing applications are provided, which can more accurately select the video playback platform and the information retention mini-program template corresponding to the target video from multiple candidate playback platforms and multiple candidate information retention mini-program templates.

[0125] After selecting the video playback platform and the information retention mini-program template corresponding to the target video, the device identifier of the terminal device, the target video, and the mini-program identifier corresponding to the information retention mini-program template can be sent to the video playback platform. Correspondingly, different mini-program codes and the correspondence between the mini-program code and the mini-program identifier can be stored on the video playback platform, and then based on the correspondence, the mini-program code corresponding to the mini-program identifier can be determined, and the mini-program code can be integrated into the preset mini-program shell to obtain the target mini-program, thereby mounting the target mini-program on the published target video. Among them, the mini-program shell configured on the video playback platform can be understood as a lightweight application container, which can be used as a basic framework to provide a running environment for the mini-program code. The mini-program codes running in the mini-program shell are different, and the resulting mini-programs are also different.

[0126] Mounting the target mini program on the published target video can be understood as mounting the target mini program on the video playback page of the published target video. For example, a mini program launch control is added to the video playback page of the target video, and the first user can click the mini program launch control to open the target mini program.

[0127] Among them, the target mini program can be bound to the device identifier of the terminal device, so that it can be used to collect the information left by the first user who watches the target video and feed it back to the terminal device; in other words, the target mini program needs to determine the return of the information left to the terminal device corresponding to the device identifier based on the device identifier.

[0128] The information left by the first user who watches the target video refers to the personal information provided by the first user through the target mini program during the process of watching the target video. For example, assuming that the target mini program is obtained based on the mini program identifier corresponding to the information left by the mini program template L1, the first user can see the following after opening the target mini program: Figure 4a or Figure 4b The mini program interface is displayed, where you can fill in your contact information, required services, clothing type, desired laundry style, and door-to-door service address as your information.

[0129] It should be emphasized that since the mini-program code integrated into the mini-program shell by the video playback platform is determined based on the information retention mini-program template selected by the terminal device, and different information retention mini-program templates correspond to different business types, the integrated target mini-program is not only more in line with the user's mini-program mounting needs, but also more adaptable to the business type to which the target object belongs, so as to more accurately collect information retention information. In this way, the playback platform and mini-program template can be determined more accurately, and the target mini-program can be mounted on the playback platform based on the mini-program template to more accurately collect information retention information, so as to more efficiently generate and publish the target video.

[0130] The above video processing method will be further explained below in combination with actual scenarios.

[0131] In an actual scenario, when the identification information, static description information and editing actions of the first time axis are encapsulated to obtain structured editing data according to the set structured data structure, an optional implementation of the set structured data structure is as follows:

[0132] {

[0133] Type: refers to the type identifier of the original material;

[0134] Id: refers to the identity of the current timeline;

[0135] Name: the name of the current timeline (i.e. the first timeline in the previous text);

[0136] Start: The starting time of the current timeline.

[0137] End: The end time of the current timeline.

[0138] frameCount: current video frame rate, that is, how many pictures are played per second;

[0139] Source: stores resources such as audio and video URLs (Uniform Resource Locator) or binary bytes;

[0140] callBackOperations: the type of editing operation;

[0141] }

[0142] The above is merely an exemplary description of the structured data structure and does not limit the specific implementation of the structured data structure.

[0143] In actual scenarios, when a user initiates a material upload operation through the upload portal in the third display area, the material link corresponding to the uploaded material can be uploaded through the upload portal; correspondingly, if the uploaded material is a video, the video link can be parsed to obtain the key frame and related information of the video, such as name, format, clarity, etc. In the process of generating the timeline of the material, the key frame and related information of the video can be displayed on the timeline of the material, so that the user can more intuitively view the material information corresponding to the timeline. If the uploaded material is audio, the audio link can be parsed to obtain the sound length and sound wave of the audio. In the process of generating the timeline of the material, the sound length and sound wave of the video can be displayed on the timeline of the material, so that the user can more intuitively view the material information corresponding to the timeline.

[0144] In actual scenarios, when a user selects a time axis in the second display area, the time period corresponding to the time axis can be highlighted on the horizontal time base line. The highlighting method can be highlighting or color changing, etc., which is not limited in this embodiment.

[0145] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 11 to 15 can be device A; for another example, the execution subject of steps 11 to 13 can be device A, and the execution subject of steps 14 to 15 can be device B; and so on.

[0146] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0147] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.

[0148] Figure 6 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, and the electronic device is applicable to the video processing method provided by the aforementioned embodiment, such as Figure 6 As shown, the electronic device may include: a memory 601 , a processor 602 , and a communication component 603 .

[0149] The memory 601 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0150] The processor 602 is coupled to the memory 601 and is used to execute the computer program in the memory 601, so as to: display the service webpage, the service webpage including a first display area corresponding to the video editor, and a second display area corresponding to the timeline manager; the first display area is used to display the original material to be edited, and the second display area is used to display the timeline corresponding to the original material, and different types of original materials correspond to different timelines; in response to the material loading operation, the original material is loaded and displayed in the first display area; and in the case that the timeline corresponding to the original material does not exist in the second display area, a timeline is generated according to the type of the original material, and the timeline is displayed In the second display area; calling the timeline manager, responding to the editing operation on the first timeline in the second display area, generating structured editing data, and transmitting the structured editing data to the video editor, the structured editing data including identification information of the first timeline, static description information corresponding to the first original material corresponding to the first timeline, and editing actions on the first original material; calling the video editor to perform editing operations on the first original material according to the static description information and editing actions in the structured editing data, and displaying the editing status and editing results of the first original material in the first display area, and generating a target video according to the editing results of multiple original materials.

[0151] Optionally, the service webpage also includes a third display area; the third display area is used for the user to load multiple original materials to be edited; the processor 602 responds to the material loading operation, and when loading the original material, is specifically used to: respond to the material upload operation initiated by the user through the upload entrance in the third display area, and use the uploaded material as the original material; and / or respond to the user's selection operation of the candidate material displayed in the third display area, and use the selected candidate material as the original material.

[0152] Optionally, when the processor 602 generates a timeline according to the type of the original material, it is specifically used to: generate track lines according to a default timeline style; determine the background color of the track lines according to the type of the original material; add the background color to the track line axis to obtain a timeline; wherein different types of original materials correspond to different background colors.

[0153] Optionally, when the processor 602 displays the timeline in the second display area, it is specifically used to: determine a first position corresponding to the horizontal time baseline of the second display area based on the start time of the original material; determine a second position corresponding to the timeline on the vertical position baseline of the second display area based on the hierarchical relationship or positional relationship between the original material and the existing original material in the target video; use the intersection of the first position and the second position as the starting position of the timeline, and display the timeline in the second display area; wherein the positional relationship of each timeline in the second display area on the vertical position baseline represents the hierarchical relationship or positional relationship of the corresponding original materials in the target video.

[0154] Optionally, when the processor 602 generates structured editing data in response to an editing operation on the first timeline in the second display area, it is specifically used to: highlight the first timeline in response to the editing operation on the first timeline; generate an editing action for the first original material based on the type information of the editing operation; generate static description information corresponding to the first original material based on the editing result information of the first timeline; and encapsulate the identification information of the first timeline, the static description information and the editing action according to a set structured data structure to obtain the structured editing data.

[0155] Optionally, when the processor 602 generates an editing action for the first original material based on the type information of the editing operation, it is specifically used to: respond to the editing operation on the first timeline, determine the type of the editing operation based on the trajectory information of the editing operation or the editing tool used; determine the editing action for the first original material based on the type of the editing operation and in combination with the correspondence between the editing operation type and the editing action.

[0156] Optionally, when the processor 602 responds to an editing operation on the first timeline and highlights the first timeline, it is specifically used to: highlight the editing effect on the first timeline according to the type of the editing operation, different editing operations correspond to different editing effects; and transmit the editing effect to the video editor so that the video editor can synchronously display the editing effect during the editing process of the first original material.

[0157] Optionally, the editing operation on the first timeline includes: a drag operation, a delete operation, a crop operation, a selection operation, an object adding operation or a zooming operation; wherein the object adding operation refers to adding a target object at a specified position of the first timeline.

[0158] Optionally, the processor 602 is also used to: when the editing operation is a drag operation, monitor whether the start time or end time of the first time axis overlaps with the end time or start time of other time axes based on the trajectory information of the drag operation; when overlap occurs, display an adsorption effect in the overlapping position or related area of ​​the two overlapping time axes, and provide prompts through auxiliary lines.

[0159] Optionally, the processor 602 is further used to: display a drag control at the starting position of the first timeline, the drag control being linked to the first timeline; and when the editing operation is a drag operation, respond to the drag operation on the drag control and initiate a drag operation on the first timeline.

[0160] Optionally, the multiple original materials include a promotional video generated based on a video production service provided by a service platform, and the target video is a result video of editing the promotional video.

[0161] Optionally, processor 602 is also used to: respond to a video generation instruction for a target object, determine the object description information and video script type of the target object; determine the script structure corresponding to the video script type; the script structure includes multiple storyboards; use an artificial intelligence big model to generate the script content of each storyboard based on the object description information and the script structure; the script content includes spoken text and storyboard description information; the storyboard description information is used to describe the picture content of the storyboard; convert the spoken text corresponding to each storyboard into spoken audio; search from the material library for a material video that matches the storyboard description information of each storyboard and the target object; synthesize the spoken audio of each storyboard with the corresponding material video into a storyboard video; according to the script structure, synthesize the storyboard videos of the multiple storyboards to obtain a promotional video for the target object.

[0162] Optionally, processor 602 is also used to: respond to the user's video publishing needs, select a target playback platform and a target information retention mini-program template from multiple candidate playback platforms and multiple candidate information retention mini-program templates; different information retention mini-program templates correspond to different business types and mini-program identifiers; send the identifier of the service platform, the target video, and the mini-program identifier corresponding to the target information retention mini-program template to the target playback platform, so that the target playback platform can publish the target video; wherein the published target video is mounted with a target mini-program, and the target mini-program is obtained by the target playback platform integrating the mini-program code corresponding to the mini-program identifier into a preset mini-program shell, and the target mini-program is bound to the identifier of the service platform, and is used to collect the information retention information of the first user who watches the target video, and feed it back to the service platform.

[0163] Further, if Figure 6 As shown, the electronic device also includes: a display 604, a power supply component 605, an audio component 606 and other components. Figure 6 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 6 Components shown.

[0164] The embodiment of the present application also provides a computer-readable storage medium, which enables the processor to implement the steps in the video processing method when the computer program is executed by the processor.

[0165] An embodiment of the present application further provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the steps in the video processing method are executed.

[0166] In this embodiment, the service platform can provide a service webpage, and the service webpage has a timeline manager and a video editor embedded therein; the user can perform timeline editing on the original material based on the display area corresponding to the timeline manager, thereby synchronously editing the original material in the display area corresponding to the video editor. On the one hand, the user can perform video editing more efficiently through the service webpage provided by the service platform, reducing the performance requirements for the terminal device; on the other hand, through the cooperation between the timeline manager and the video editor, the user can perform timeline editing more efficiently based on the timeline manager, so as to synchronize the editing operation more efficiently in the display area corresponding to the video editor, and more intuitively view the editing status and editing results of the edited original material, thereby improving the efficiency of video editing.

[0167] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0168] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0169] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0170] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0171] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0172] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0173] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0177] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0180] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A video processing method, characterized in that: The method is applied to a service webpage provided by a service platform, wherein the service webpage has a timeline manager and a video editor embedded therein, and the timeline manager and the video editor interact with each other using structured data; wherein the service webpage runs on a terminal device, and the method comprises: Displaying the service webpage, wherein the service webpage includes a first display area corresponding to the video editor and a second display area corresponding to the timeline manager; The first display area is used to display the original material to be edited, and the second display area is used to display the time axis corresponding to the original material, and different types of original materials correspond to different time axes; In response to the material loading operation, loading the original material, and displaying the original material in the first display area; and If the time axis corresponding to the original material does not exist in the second display area, generating a time axis according to the type of the original material, and displaying the time axis in the second display area; Calling the timeline manager, responding to the editing operation on the first timeline in the second display area, generating structured editing data, and transmitting the structured editing data to the video editor, wherein the structured editing data includes identification information of the first timeline, static description information corresponding to the first original material corresponding to the first timeline, and editing actions on the first original material; The video editor is called to perform editing operations on the first original material according to the static description information and editing actions in the structured editing data, and the editing status and editing results of the first original material are displayed in the first display area, and a target video is generated according to the editing results of multiple original materials.

2. The method according to claim 1, characterized in that The service webpage further includes a third display area; the third display area is used for allowing the user to load a variety of original materials to be edited; Respond to the material loading operation and load the original material, including: In response to a material upload operation initiated by a user through an upload entry in the third display area, the uploaded material is used as the original material; and / or In response to a user's selection operation on the candidate material displayed in the third display area, the selected candidate material is used as the original material.

3. The method according to claim 1, characterized in that Generate a timeline according to the type of the original material, including: Generate track lines according to the default timeline style; Determining the background color of the track line according to the type of the original material; The background color is added to the track line axis to obtain a time axis; wherein different types of original materials correspond to different background colors.

4. The method according to claim 1, characterized in that: Displaying the timeline in the second display area includes: Determine, according to the start time of the original material, a first position corresponding to the second display area on the horizontal time base line; Determine, according to a hierarchical relationship or a positional relationship between the original material and the existing original material in the target video, a second position corresponding to the timeline on the vertical position baseline of the second display area; Using the intersection of the first position and the second position as the starting position of the time axis, and displaying the time axis in the second display area; The positional relationship of each time axis in the second display area on the vertical position baseline represents the hierarchical relationship or positional relationship of each corresponding original material in the target video.

5. The method according to any one of claim 1, characterized in that: In response to an editing operation on the first time axis in the second display area, generating structured editing data, including: In response to an editing operation on the first timeline, highlighting the first timeline; generating an editing action for the first original material according to the type information of the editing operation; Generate static description information corresponding to the first original material according to the editing result information of the first timeline; According to the set structured data structure, the identification information of the first time axis, the static description information and the editing action are encapsulated to obtain the structured editing data.

6. The method according to claim 5, characterized in that Generating an editing action for the first original material according to the type information of the editing operation includes: In response to an editing operation on the first timeline, determining a type of the editing operation according to track information of the editing operation or an editing tool used; The editing action for the first original material is determined according to the type of the editing operation and in combination with the corresponding relationship between the editing operation type and the editing action.

7. The method according to claim 5, characterized in that In response to the editing operation on the first timeline, highlighting the first timeline includes: According to the type of the editing operation, highlighting the editing effect on the first timeline, where different editing operations correspond to different editing effects; and The editing effect is transmitted to the video editor so that the video editor can synchronously display the editing effect during the editing process of the first original material.

8. The method according to any one of claims 1 to 7, characterized in that: The editing operation on the first timeline includes: a drag operation, a delete operation, a crop operation, a selection operation, an object adding operation or a zoom operation; wherein the object adding operation refers to adding a target object at a specified position of the first timeline.

9. The method according to claim 8, characterized in that Also includes: In the case where the editing operation is a drag operation, monitoring whether the start time or the end time of the first time axis coincides with the end time or the start time of other time axes according to the track information of the drag operation; When overlap occurs, an adsorption effect is displayed at the overlap position or related area of ​​the two overlapping timelines, and prompts are given through auxiliary lines.

10. The method according to claim 9, characterized in that Also includes: Displaying a drag control at the start position of the first time axis, wherein the drag control is linked to the first time axis; When the editing operation is a drag operation, in response to the drag operation on the drag control, a drag operation is initiated on the first timeline.

11. The method according to any one of claims 1 to 7, characterized in that: The multiple original materials include a promotional video generated based on a video production service provided by a service platform, and the target video is a result video of editing the promotional video.

12. The method according to claim 11, characterized in that Also includes: In response to a video generation instruction for a target object, determine object description information and a video script type of the target object; Determine a script structure corresponding to the video script type; the script structure includes a plurality of storyboards; Generate the script content of each storyboard based on the object description information and the script structure using the artificial intelligence big model; the script content includes the spoken copy and the storyboard description information; the storyboard description information is used to describe the screen content of the storyboard; Convert the spoken text corresponding to each storyboard into spoken audio; Searching for a material video that matches the storyboard description information of each storyboard and the target object from a material library; The spoken audio of each storyboard is combined with the corresponding source video into a storyboard video; According to the script structure, the storyboard videos of the plurality of storyboards are synthesized to obtain a promotional video for the target object.

13. The method according to any one of claim 11, characterized in that: Also includes: In response to the user's video publishing needs, a target playback platform and a target retention mini program template are selected from multiple candidate playback platforms and multiple candidate retention mini program templates; Different retention mini program templates correspond to different business types and mini program logos; Sending the service platform identifier, the target video, and the mini-program identifier corresponding to the target information-reservation mini-program template to the target playback platform, so that the target playback platform can publish the target video; Among them, the published target video is mounted with a target mini-program, and the target mini-program is obtained by the target playback platform integrating the mini-program code corresponding to the mini-program identifier into a preset mini-program shell. The target mini-program is bound to the identifier of the service platform, and is used to collect the retention information of the first user who watches the target video and feed it back to the service platform.

14. An electronic device, characterized in that: include: A memory and a processor; wherein the memory is used to: store one or more computer instructions; The processor is configured to execute the one or more computer instructions to perform the steps of the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The method comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps in the method according to any one of claims 1 to 13 are performed.

Images